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ABSTRACT 


RADC  Reliability  Notebook,  Volume  I,  is  an  updating  of  the  RADC 
Reliability  Notebook  which  was  first  published  in  1958  and  which  had 
been  revised  several  times  up  until  the  Fall  of  1956.  This  updating 
has  resulted  in  a  completely  updated  (except  for  Section  8)  notebook 
in  arrangement,  format  and  material  as  per  the  contract  under  which 
the  effort  was  conducted.  There  are  twelve  chapters  comprising, 
first,  a  general  discussion,  followed  by  a  presentation  of  information 
which  project  managers  and  project  engineers  can  use  to  be  more  ef¬ 
fective  in  predicting,  measuring  and  improving  system  and  equipment 
reliability  A  subject  index  has  been  included  at  the  end  in  order  to 
provide  the  user  with  a  guide  to  locating  specific  information. 

This  updating  was  based  on  a  major  collection  of  existing  information 
with  emphasis  on  reliability  in  large  system  developmei  as  well  as  in 
non-system  or  off-the-shelf  hardware  procurement  programs.  Empha¬ 
sis  has  been  placed  on  prediction  techniques;  test  demonstration  plans 
and  analysis  of  test  data;  and  on  the  relationship  between  reliability 
and  various  other  factors  including  engineering  disciplines,  program 
milestones,  design  reviews  and  engineering /acceptance  tests  covered 
at  length  in  various  Air  Force  documents  such  as  AFSCM/AFLCM 
310-1  and  the  AFSCM  375  series  of  publications.  Of  particular  signif¬ 
icance  is  the  inclusion  of  information  on  Bayesian  statistics  and  the  ap¬ 
plication  of  this  statistical  concept  to  the  development  of  demonstration 
test  plans  and  the  interpretation  of  test  data. 

The  information  presented  in  this  updated  version  of  the  RADC  Reliability 
Notebook,  together  with  the  references  and  the  guidelines  contained  in  Air 
Force  program  management  publications,  will  provide  project  engineers 
and  project  managers  with  a  sound  basis  for  implementing  reliability  ori¬ 
ented  effort  and  program  plans  and  for  monitoring  to  insure  that  reliabil¬ 
ity  objectives  will  be  met. 
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?tem  project  officers,  reliability  promra.m  manamars ,  ind  other':  who  arc 
'or  direct  ins  system  ana  equipment  reliability  pro'rra.ms  have  lorv" 
hoi’-’  need  For  a  reliability  handbook  oriented  toward:;  reliability 
•T.q  -  am  monay ament  with  attendant  coverage  of  the  tec.'inical  disci.nl  ines  of 
rvl lability  engineerin'1.  Tne  objeetivo  of  this  effort  was  the  -’evelonment  of 
•>urh  a  document ,  Volume  T  of  the  PARC  Reliabi]  ity  Notebook. 


1.  rr?rs  Volume  T  is  designed  to  nrid,Te  the  cyjo  whicn  nres-°ntl”  -,xists  between 
t:v;  '  .rv.li.ar  re]  iabil itv  textbooks  md  the  several  reliability  a  i.nafTo:TK:nt 
’.variant;  currently  available  such  as  rhV's  handbook  for  pclia'n i.lity  and  laintain- 
nbllity  ' foir  ions  ind  the  ‘'A* ISO  two-volume  Reliability  'lann'’emont  handbook.  The 
'^’liability  textbooks ,  of  course,  provide  reliabi lit;;  on-’ insert™  Hmdament a.1  s 
and  usurdi.lv  include  charters  on  reliabi ]  it-/  prediction,  improvement  md 
"v?  i--un?.'ier.t  tec  uniques.  The  reliability  •aanamenent  documents  contain  material 
’hi.cu  is  nri’Vrtly  a ’mi n Istrativ?  and  include  discuss’ ons  oF  tie  nnnnrrerial- 
oriented  re l.i  ibi lit-'  activities  required  dnrin"  system  development.  heather, 
i .-/ever,  n-i:e  any  attempt  to  deseri.lxe  tae  relationship  between  the  two  subject 
ureas,  nor  do  they  convey  tne  dependence  of  one  on  the  other.  Volume  T  not  only 
contain?  loth  administrative  and  technical  reliability  information,  but  also  combines 
tun  nnT’fimi]  -oriented  reliability  activities  with  the  technioal-o^iented. 
i.nfr  rail  ion  neccssarv  for  making  decisions  on  reliah’lity  ra  purements ,  predictions 
in.i  tests.  Tius,  cor  example,  the  reliability  orrmnm  nonater  is  not  only  able  to 
iota  trine  at  '/hat  st me  or  development  a  reliability  prediction  is  required,  but  he 
L.s  also  are  sente/.,  i  detailed  account  or  tne  sneuifio  reliabi  1.5. tv  prediction 
techniques  which  he  can  use  at  that  tire. 


3.  The  RAPC  f.eli.auil ity  ’’otelxxak,  ThhC-Tr’.-57-l 'J8 ,  now  consists  of  two  volumes  - 
this  Volume  1  .and  Voluse  TT  (DOC  ’<o.  /W821G40)  which  contains  undated  part 
failure  rate  data  and  mli  air  litv  mmdiction  mo ’oil  3,  Mthouah  the  material,  in 
Volume  T  is  not  su, eject  to  -as  rapid  change  -as  that  in  Volume  IT,  revisions  and 
additions  /ill  l>=  made  when  required. 
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CHAPTER  1 


INTRODUCTION 


System  Reliability  has  become  a  key  factor  in  describing  tactical/ 
operational  needs  and  in  turn  it  has  become  a  key  characteristic  in 
evaluating  system  requirements  and  resultant  performance.  Tactical 
requirements  have  become  such  that  systems  are  becoming  more  com¬ 
plex.  As  a  result,  reliability  requirements  become  more  difficult  to 
meet  from  a  performance  point  of  view  as  well  as  from  a  dollar  and 
schedule  point  of  view.  In  recognition  of  this  it  has  become  increasingly 
more  apparent  that  the  positive  application  of  reliability  engineering 
technique15  and  the  monitoring  of  reliability  performance,  from  incep¬ 
tion  through  delivery  and  follow-up  for  improvement,  can  be  very  ef¬ 
fective  in  helping  to  ensure  that  system  reliability /performance  require¬ 
ments  will  be  met. 

t 

As  with  other  performance  characteristics,  reliability  is  a  quantitative 
element  and,  as  such,  it  can  be  specified  as  a  requirement,  it  can  be 
predicted  based  on  available  design  information,  it  can  be  measured 
through  test  and  performance  data,  and  it  can  be  observed  under  oper¬ 
ation  conditions.  Because  of  its  quantitative  nature,  its  predictability 
and  measurability,  reliability  can  be  monitored  and  influenced  at  various 
stages  in  the  system  life  cycle.  Proper  recognition  of  these  features  and 
the  application  of  resources  to  them,  both  from  an  emphasis  and  timing 
point  of  view,  can  significantly  influence  the  potential  for  meeting  reli¬ 
ability  requirements  within  cost  and  schedule. 

Use  of  these  features  and  the  application  of  resources  oriented  toward 
monitoring,  controlling,  and  influencing  reliability  must  be  integrated  with 
various  other  activities  required  as  part  of  a  system  development  or  equip¬ 
ment  procurement  program.  While  reliability  is  strongly  influenced  by 
individual  designers,  the  responsibility  for  the  direction  and  emphasis,  for 
developing  assurances,  and  planning  and  devoting  resources  required  to 
effectively  monitor  and  influence  reliability  rests  with  system  project  offi¬ 
cers,  reliability  managers,  an  1  those  who  are  responsible  for  directing  sys¬ 
tem  and  equipment  reliability  programs.  The  amount  of  information  and 
knowledge  required  to  make  sound  decisions  in  fulfilling  these  responsibilities 
is  vast  and  covers  a  wide  range.  This  range  includes  administrative  oriented 
techniques,  information  and  requirements  for  miaking  evaluations,  implement¬ 
ing  activities  and  receiving  information  as  well  as  technically  oriented  inform¬ 
ation  necessary  for  making  decisions  covering  requirements,  predictions  and 
tes  ts . 
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This  handbook  provides  information  in  the  administrative  areas  as 
well  as  in  the  technical  area  pertinent  to  guiding  its  users  toward  reli¬ 
ability  planning  and  making  the  reliability -oriented  evaluations  and  de¬ 
cisions  which  will  provide  a  greater  degree  of  assurance  that  reliability 
requirements  will  be  met.  It  is  designed  to  aid  management  in  reaching 
decisions  concerning  reliability  aspects  of  a  program  throughout  the 
system  development  cycle  or  equipment  procurement  cycle.  It  outlines 
techniques  and  practices  as  well  as  Air  Force  requirements  in  such  a  w<ay 
as  to  provide  guidelines  which  can  be  used  for  making  decisions  concern¬ 
ing  the  planning  and  implementing  of  pertinent  activities. 

The  first  seven  chapters  of  this  Volume  l*.of  the  RADC  Reliability  Note¬ 
book  is  primarily  management  or  administrative  oriented.  The  information 
covers  reliability  program  management,  reliability  engineering  management, 
data  management,  assuring  reliability  program  effectiveness,  and  field  data. 
There  is  considerable  tie-in  to  Air  Force  documents  (which  detail  various 
system  program  requirements)  in  such  a  way  that  reliability  can  become 
an  integral  part  of  a  system  program  within  existing  regulations  and  require¬ 
ments.  Although  not  altogether  exhaustive  since  such  a  treatment  would 
become  extremely  voluminous,  the  information  docs  provide  the  guidance 
needed  to  plan  and  prepare  for  these  reliability  oriented  activities,  controls, 
and  data  receivables  which  meet  the  requirements  of  the  program  at  hand. 

Chapters  8  through  11  are  more  technically  oriented  from  the  point  of  view 
of  the  application  of  techniques.  The  information  in  these  latter  chapters 
covers  allocation,  prediction,  measurement,  and  improvement.  There  are 
examples  which  can  be  used  as  guides  in  applying  the  techniques  to  specific 
problems.  Here  again,  the  information  is  not  completely  exhaustive  and 
does  not  cover  extremely  complex  models.  The  coverage,  however,  is  suf¬ 
ficient  to  provide  the  user  with  the  tools  and  information  needed  to  apply  the 
techniques  to  certain  classes  of  problems,  to  understand  the  elements  and 
parameters  of  significance,  and  to  develop  an  understanding  of  the  approaches 
which  must  be  undertaken  in  certain  instances  in  order  to  develop  meaningful 
results. 

The  final  chapter  contains  a  bibliography  which  can  be  used  to  supplement 
the  information  in  this  volume  and  which  can  be  used  as  a  source  of  more 
detailed  data  with  respect  to  each  of  the  topics  contained  in  the  Notebook. 


*  Volume  2  of  the  RADC  Reliability  Notebook  contains 
detailed  failure  rate  data. 
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RELIABILITY  CONCEPTS 

RELIABILITY  IN  SYSTEM  DEVELOPMENT 

In  recent  years  there  has  been  a  continuous  demand  for  more  extensive 
and  sophisticated  hardware  systems  to  meet  national  defense  require¬ 
ments.  This  demand  has  been  accompanied  by  advancements  in  analytic 
techniques  for  determining  defense  needs  and  for  taking  an  integrated 
systems  approach  to  meeting  defense  objectives.  Advancements  in 
these  analytic  techniques  have  shown  that  an  integrated  systems  ap¬ 
proach  to  hardware  development  and  procurement  is  often  required  in 
order  to  meet  performance  goals  within  specified  schedule,  finance, 
and  resource  constraints.  Without  an  integrated  approach,  standardi¬ 
zation  objectives  may  not  be  met;  systems  may  be  designed  which  re¬ 
quire  maintenance  and  operational  talents  far  exceeding  those  required 
to  be  compatible  with  other  system  segments  or  performance  needs; 
and  interface  problems  might  be  created  which  could  seriously  affect 
the  capability  to  fully  utilize  system  characteristics.  The  realization 
that  these  and  other  undesirable  effects  could  be  more  nearly  corrected 
through  an  integrated  approach  to  development  and  procurement  together 
with  the  development  of  more  sophisticated  analytic  techniques  and  the 
means  for  practically  applying  them  has  been  a  key  force  in  motivating 
the  Department  of  Defense  and  the  Air  Force  toward  a  total  systems 
approach.  As  one  means  of  implementing  this  approach,  the  Air  Force 
has  developed  a  series  of  publications  (e.  g.  ,  the  AFSCM  375-series) 
designed  to  provide  the  Systems  Project  Office  (SPO),  systems  program 
managers  and  project  engineers  with  guidelines,  recommendations  and 
requirements  to  help  ensure  that  a  systems  approach  is  being  taken 
during  each  of  the  development  and  life  cycle  phases  and  to  help  with 
the  decision  making  and  information  llow  processes.  These  publications 
cover  a  wide  variety  of  organization,  technical,  procedural,  reporting 
and  data  flow  recommendations  and  requirements  which  system  managers 
and  project  engineers  are  expected  to  follow  in  order  to  achieve  procure¬ 
ment  and  performance  objectives.  In  certain  instances  particular  disci¬ 
plines  require  additional  emphasis  because  they  are  significant  in  terms 
of  their  impact  on  system  performance.  One  such  discipline  is  relia¬ 
bility  in  terms  of  reliability  program  and  engineering  management  in¬ 
formation  and  reliability  engineering  techniques  and  the  significant 
impact  these  activities  can  have  on  system  reliability  as  a  performance 
characteristic . 

In  reality,  reliability  as  a  performance  characteristic  has  been  a  key 
factor  in  systems  and  equipment  development  and  procurement.  How¬ 
ever,  demands  have  been  oriented  toward  more  complex  versatile 
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systems  and  increasingly  higher  levels  of  reliability  with  each  new 
generation  of  weapon  or  communication  system.  For  example,  during 
the  1  950' s  an  MTBF  of  50  hours  was  often  acceptable  for  a  system  of 
moderate  complexity.  By  the  early  1  9  6  0 1  s  ,  however,  this  value  had 
increased  to  the  order  of  100  to  500  hours  for  systems  of  equivalent  or 
even  higher  complexity.  Currently,  an  MTBF  requirement  of  1000 
hours  is  not  uncommon,  and  within  another  decade  an  increase  to 
1  0,  000  hours  or  higher  is  conceivable.  This  demand  for  increasingly 
higher  levels  of  reliability  has  been  taking  place  while  system  functional 
complexity  also  has  been  increasing  at  a  comparable  rate.  This  contin¬ 
uing  demand  for  incre  ased  reliability  and  system  complexity  has  resulted 
in  increased  emphasis  on  reliability  engineering,  management,  program, 
and  analytic  technique .•■  for  meeting  these  objectives. 

One  approach  to  meeting  the  requirement  for  increased  emphasis  on 
reliability  oriented  efforts  is  to  provide  guidelines  and  information 
which  can  be  used  as  a  basis  for  implementing  organizational  structures 
of  which  reliability  is  an  integral  part,  directing  reliability  oriented 
technical  efforts  and  analyses,  and  evaluating  periodic  and  final  results 
to  ensure  that  reliability  as  well  as  other  important  technical,  perfor¬ 
mance,  and  physical  characteristics  and  goals  have  been  met. 

This  Volume  I  of  the  RADC  Reliability  Notebook  has  jeen  developed  with 

that  specific  purpose - to  provide  the  System  Project  Office  (SPO)  and 

System  Program  Directors  with  guidelines  which  will  help  ensure  that 
reliability  performance  objectives  for  large  scale  complex  systems  will 
be  met  and  to  provide  information  which  will  permit  application  of  the 
guidelines  to  smaller  scale  programs  as  well.  The  basic  philosophy  of 
Volume  I  is  oriented  toward  application  of  techniques  and  effort  required 
to  establish  meaningful  reliability  objectives  commensurate  with  tactical 
and  performance  needs  and  to  meet  these  objectives  within  resource, 
financial  and  schedule  constraints.  The  concepts  and  information  pre¬ 
sented  are  directed  toward  two  basic  disciplines.  Reliability  Assurance 
and  Reliability  Achievement.  Reliability  assurance  includes  activities 
directed  toward  establishing  appropriate  reliability  development  goals, 
monitoring  program  activities,  and  evaluating  results  to  verify  that 
established  goals  are  reached.  Reliability  achievement  includes  applica¬ 
tion  of  reliability  engineering  techniques  performed  for  the  specific  pur¬ 
pose  of  achieving  the  required  level  of  reliability. 


REDIABILITY  ASSURANCE 

Reliability  assurance,  i.  e.  ,  reliability  program  functions  that  are  per¬ 
formed  for  the  explicit  purpose  of  assuring  that  a  required  level  of  relia¬ 
bility  is  achieved,  is  probably  the  most  important  function  of  the  reliability 
group  of  the  system  program  office.  Reliability  assurance  involves  a 
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variety  of  activities  that  can  he  classified  under  four  general  functions: 
allocation,  specification,  prediction  and  demonstration.  The  various 
reliability  assurance  fvinctions  typically  occur  during  different  phases 
of  the  system  life  cycle. 

Reliability  allocation  is  the  process  of  establishing  reliability  require¬ 
ments  for  various  subdivisions  of  a  system  based  on  a  previously  es¬ 
tablished  overall  system  reliability  goal.  Overall  system  reliability 
requirements  typically  are  derived  from,  the  operational  requirements 
and  constraints  of  the  mission.  Once  this  level  has  been  established, 
a  number  of  interrelated  factors  such  as  importance  or  criticality, 
and  complexity  of  individual  functions  are  weighed  against  state-of-the- 
art  limitations  and  various  design  constraints  to  arrive  at  compatible 
and  practical  levels  of  reliability  for  each  defined  subdivision  of  the 
system . 

The  second  major  activity  of  a  reliability  assurance  program  is  that  of 
developing  the  reliability  requirements  for  the  system  and  detail  speci¬ 
fications.  The  general  System  Specification  which  contains  the  technical 
requirements  for  the  system  as  an  entity,  is  usually  prepared  by  the  pro¬ 
curement  organization,  and  provides  the  basic  technical  requirements 
governing  the  contract  definition  activities.  The  System  Specification 
contains  reliability  requirements  that  must  be  stated  in  quantitative 
terms.  The  preliminary  reliability  allocation  activities  provide  the 
principle  input  to  the  development  of  the  system  reliability  requirements. 

In  fact,  a  reliability  apportionment  model  supporting  the  allocation  of 
reliability  values  assigned  to  system  segments  is  recommenued  as  a 
part  of  the  reliability  requirements  paragraph  of  the  System  Specification. 

The  general  reliability  requirements  of  the  System  Specification  are  sub¬ 
sequently  refined  and  expanded  during  the  preparation  of  the  Detail  Speci¬ 
fication.  These  specifications  include  requirements  peculiar  to  the  design, 
development,  test,  and  qualification  of  individual  contract  end  items,  and 
includes  specific  reliability  requirements  stated  in  appropriate  quantita¬ 
tive  terms . 

One  major  reliability  oriented  activity  during  system  development  is 
that  of  assuring,  with  an  acceptable  level  of  confidence,  that  the  speci¬ 
fied  reliability  requirements  are  being  met  before  the  design  has  pro¬ 
gressed  to  the  point  that  changes  are  impractical.  This  portion  of  the 
reliability  program  involves  application  of  techniques  of  reliability  pre¬ 
diction,  i.e.,  estimating  the  probable  level  of  reliability  that  will  be 
achieved  based  on  characteristics  of  the  system  design. 

The  results  of  a  reliability  prediction  provide  quantitative  information 
concerning  the  probable  level  of  achieved  reliability,  help  to  identify 
weak  or  problem  areas  in  the  design,  and  provide  a  quantitative 
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evaluation  of  proposed  design  changes.  Another  important  use  of  relia¬ 
bility  prediction  is  in  performing  reliability  analyses  for  use  in  design 


reviews 


Once  a  design  is  accepted,  and  as  end  items  are  produced,  the  achieved 
level  of  reliability  is  demonstrated  as  a  part  of  the  acceptance  tests. 

The  objective  of  reliability  demonstration  is  to  obtain  quantitative  empir¬ 
ical  evidence  that  the  hardware  is  in  compliance  with  specified  reliability 
requirements.  The  demonstrations  are  conducted  in  accordance  with  an 
approved  test  plan  which  includes  a  statistically  designed  procedure 
specifying  test  duration,  test  conditions,  sample  size  and  acceptance 
criteria. 


RELIABILITY  ACHIEVEMENT  TECHNIQUES 


Reliability  achievement  includes  reliability  engineering  and  improve¬ 
ment  activities  that  are  performed  throughout  the  system  life  cycle, 
and  which  influence  system  design.  Several  distinctive  techniques  or 
engineering  methodologies  include  derating,  redundancy,  simulation, 
and  data  feedback  and  analysis. 


Part  derating  is  a  design  technique  used  to  reduce  the  probability  of 
failure  of  parts  in  a  particular  design.  Through  this  technique,  a 
safety  factor  is  established  by  selecting  parts  capable  of  withstanding 
stresses  in  excess  of  those  likely  to  be  encountered  during  operation. 
Thus,  failures  that  result  from  normal  variations  in  operational 
stresses  can  be  significantly  reduced. 


Redundancy  is  the  technique  of  providing  alternate  devices  or  methods 
for  performing  a  given  function  when  the  primary  device  or  method  has 
failed.  In  some  cases  a  complete  standby  system  is  d\iplicated,  while 
in  other  cases  redundancy  can  be  appliea  to  equipments,  units,  or  even 
individual  parts.  Judicious  application  of  redundancy  can  result  in  very 
significant  improvement  in  system  reliability.  However,  size,  weight 
and  cost  restrictions  often  necessitate  careful  trade-off  analyses  in 
optimizing  the  design  approach. 


Simulation,  or  artificially  approximating  functional  characteristics, 
is  a  valuable  tool  which  has  direct  application  in  reliability  engineering. 
Two  basic  types  of  simulation  are  commonly  used  in  system  develop¬ 
ment.  Computer  simulation  of  system  functions  provides  a  method  for 
"exercising"  a  des ign  before  hardware  items  are  produced.  A  mathe¬ 
matical  model,  generated  to  describe  all  functions  and  interface  re¬ 
lationships  of  the  system,  is  programmed  for  solution  by  analog  or 
digital  computer.  Appropriate  variation  of  input  data  during  solution 
provides  dynamic  evaluations  of  the  system  design.  Such  simulation 
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can  be  used  to  evaluate  design  reliability  in  terms  of  changing  opera¬ 
tional  demands  and  intermsof  total  system  configuration  before  ex¬ 
pensive  hardware  items  are  produced. 

A  second  type  of  simulation  is  that  of  exercising  hardware  models  of 
system  components  under  simulated  operational  conditions.  A  range 
of  environmental  conditions  and  operational  stresses  are  artificially 
produced  in  a  carefully  controlled  manner.  Such  simulation  permits 
empirical  reliability  data  to  be  generated  under  precisely  known  con¬ 
ditions  and  stresses.  The  data  thus  generated  can  be  used  as  the  basis 
for  determining  the  source  of  reliability  problem  areas  and  for  de¬ 
veloping  appropriate  corrective  measures. 

Reliable  systems  are  the  result  of  mature  designs  reflecting  experience 
gained  during  successive  redesign  and  test  cycles,  and  during  the  opera¬ 
tional  phases  of  previously  developed  systems.  In  order  to  take  full 
advantage  of  such  experience,  it  is  essential  that  all  pertinent  historical 
data  be  available  to  the  design  engineer.  Therefore,  an  important 
activity  of  reliability  achievement  is  the  acquisition  of  test  and  opera¬ 
tional  data,  the  analysis  ol  these  data  to  extract  pertinent  reliability 
information,  and  the  presentation  of  such  information  to  design  engi¬ 
neering  groups  in  a  useful  form.  As  a  result  of  the  need  for  effective 
data  feedback  and  analysis  procedures  to  support  reliability  improve¬ 
ment  programs,  efficient  and  uniform  data  collection  and  analysis  pro¬ 
cedures  have  been  developed  and  are  available  to  the  reliability  engineer. 
These  procedures  are  proving  to  be  one  of  the  basic  tools  of  reliability 
improvement  activities. 

RELIABILITY  MANAGEMENT  AND  ENGINEERING 

There  are  two  general  types  of  effort  which  make  use  of  reliability 
assurance  and  achievement  disciplines.  These  are  reliability  program 
management  effort  and  reliability  engineering  effort.  The  management 
effort  is  oriented  toward  establishing  responsibilities,  planning,  and 
creating  organizational  relationships  and  toward  determining  basic  ap¬ 
proaches  for  implementing  pertinent  organizational  and  engineering 
activity  which  can  be  effective  in  monitoring  reliability  performance, 
encouraging  application  of  reliability  engineering  techniques,  and  eval¬ 
uating  results.  The  reliability  engineering  effort  is  oriented  toward 
application  of  specific  reliability  engineering  techniques,  approaches 
and  data  under  various  design,  chedule  and  development  conditions. 

Thus  management  effort  is  very  directly  related  to  assurance  activi¬ 
ties  and  engineering  effort  is  directly  related  to  achievement  activities 
though  there  is  some  overlapping  which  may  vary  from  phase  to  phase. 


Considering  lead  times,  degree  of  coordination  required  in  the  develop¬ 
ment  of  complex  systems,  and  the  increasing  importance  of  reliability 
as  a  major  objective  of  system  design,  it  is  essential  that  reliability 
efforts  be  integrated  into  the  overall  system  development  program  dur¬ 
ing  all  phases  of  the  system  life  cycle.  Specific  activities  that  are 
effective  in  ensuring  that  appropriate  reliability  goals  are  established 
and  met  vary  from  phase  to  phase.  Program  management  effort  that 
is  effective  in  assuring  reliability  during  early  phases  is  quite  different 
from  the  kind  of  effort  that  may  be  required  in  later  phases.  Relia¬ 
bility  engineering  techniques  that  are  most  appropriate  also  depend  on 
the  particular  life  cycle  phase.  Therefore  an  effective  reliability  pro¬ 
gram  should  include  provisions  for  an  appropriate  organization  and  the 
coordination  of  activities  to  provide  a  contmuous  program  that  progresses 
from  phase  to  phase  as  a  part  of  the  overall  system  development  program. 

The  techniques  and  guidelines  presented  in  this  notebook  are  related, 
where  possible  and  appropriate,  to  life  cycle  phases  of  system  develop¬ 
ment. 

5.  RELIABILITY  PROGRAM  ACTIVITIES 

Ultimate  responsibility  in  system  reliability  rests  with  the  System  Pro¬ 
gram  Director,  who  has  the  responsibility  to  see  that  the  reliability 
program  requirements,  organizational  elements,  and  resources  have 
been  appropriately  established  and  to  ensure  that  effective  reliability 
assurance  and  achievement  activities  are  performed  and  verified  at  each 
significant  program  milestone. 

Typical  reliability  management  and  reliability  engineering  assurance 
and  achievement  activities  during  the  Conceptual,  Definition,  Acquisition 
and  Operational  phases  of  a  system  life  cycle  are  summarized  below. 
These  are  discussed  in  more  detail  in  subsequent  chapters  of  this  note¬ 
book. 

5.  1  Reliability  Program  Activities  During  the  Conceptual  Phase 

The  conceptual  phase  is  tr  earliest  defined  phase  of  the  system 
life  cycle.  During  this  phase,  when  system  concepts  are  being  es¬ 
tablished,  the  role  of  the  reliability  program  is  not  always  clearly 
defined.  The  important  activities  revolve  around  interpreting  sys¬ 
tem  operational  objectives  in  terms  of  reliability  requirements, 
and  performing  initial  allocation  analysis  to  define  reliability  goals 
for  individual  subsystems. 

The  objective  of  the  reliability  program  effort  during  the  concep¬ 
tual  phase  is  to  assure  that  appropriate  and  realistic  system  and 
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subsystem  reliability  requirements  are  incorporated  in  the 
Preliminary  Technical  Development  Plan  (PTDP)  which  is  the 
basic  document  governing  the  Definition  Phase  activities. 

5.2  Reliability  Program  Activities  During  the  Definition  Phase 

The  Definition  Phase  of  the  system  iife  cycle  is  devoted  to 
translating  system  functional  requirements  generated  during  the 
conceptual  phase  into  detailed  system  and  systein  element  require¬ 
ments  that  will  govern  subsequent  acquisition  efforts.  During  this 
phase,  the  reliability  program  includes  both  assurance  and  achieve¬ 
ment  activities.  Reliability  assurance  activities  include  refinement 
of  system  reliability  allocations  to  provide  meaningful  reliability 
requirements  for  the  initial  system  specification,  and  review  and 
evaluation  of  competing  contractor's  proposals  to  assure  that  the 
reliability  requirements  will  be  met.  Reliability  achievement  or 
engineering  activities  during  this  phase  include  modeling,  predic¬ 
tions  and  trace-off  study  analyses  in  expanding  system  specifications, 
and  developing  basic  design  approaches  to  be  included  in  Acquisition 
Phase  proposals. 

5.3  Reliability  Program  Activities  During  the  Acquisition  Phase 

The  Acquis iton  Phase  is  devoted  to  development,  production,  and 
government  acceptance  of  items  on  contract.  During  this  phase, 
a  variety  of  i  ''liability  assurance  and  achievement  activities  are 
performed.  Some  of  the  more  important  reliability  achievement 
activities  during  the  acquisition  phase  include  consideration  of 
reliability  objectives  and  constraints  in  performing  design  trade-off 
analyses,  ensuring  application  of  effective  reliability  engineering 
principles  in  the  design,  (selection  of  reliable  parts,  derated  part 
applications,  redundant  configurations  and  other  reliability  design/ 
engineering  techniques)  and  implementing  design  changes  where 
necessary  to  improve  reliability.  These  activities  are  supplemented 
by  reliability  assurance  functions  such  as  imposing  reliability  re¬ 
quirements  on  subcontractors  and  vendors,  and  developing  and  im¬ 
plementing  reliability  evaluation  and  test  programs  to  assess  the 
reliability  of  the  final  product. 

5.4  Reliability  Program  Activities  During  the  Operational  Phase 

The  Operational  Phase  begins  when  the  first  contract  end  item  is 
accepted  and  turned  over  to  the  user,  and  continues  until  disposition 
of  the  system.  The  reliability  program  during  this  phase  includes 
reliability  achievement  activities  such  as  engineering  analyses  and 
development  of  reliability  improvement  nv  difi  ations.  Other  key 
reliability  activities  during  the  Operational  Phase  include  collection 
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and  analysis  of  field  failure  data,  including  data  from  reliability 
demonstrations  performed  during  operation  testing,  and  assuring 
that  modifications  introduced  for  reasons  other  than  reliability  do 
not  degrade  system  reliability. 

b.  RELIABILITY  PROGRAM  INFORMATION 

The  preceding  discussion  serves  as  an  introduction  to  the  need  for 
and  major  objective  of  the  reliability  program  during  the  system  de¬ 
velopment  cycle,  and  has  identified  certain  fundamental  reliability 
achievement  and  assurance  activities.  This  chapter  has  been  pre¬ 
sented  to  establish  a  point  of  reference  and  departure  for  the  balance 
of  the  handbook  which  contains  detailed  discussion  of  the  various  re¬ 
liability  concepts  and  program  activities  identified  herein.  A  more 
comprehensive  overview  of  tne  total  relis.bility  program  can  be  obtained 
by  reviewing  the  introductory  paragraphs  of  each  of  the  succeeding 
chapters. 

Additional  general  information  concerning  various  aspects  of  reli¬ 
ability  programs  can  be  found  in  current  literature  such  as  the  military 
documents  listed  below.  Additional  references  on  specific  subjects  are 
presented  at  the  end  of  each  chapter  where  appropriate,  and  a  complete 
bibliography  is  presented  in  Chapter  12. 

Reliability  Management  Handbook,  Arinc  Research  Inc.  Report 
No.  TOR-269  (4303)- 9,  14  February  1964.  (DDC  No.  AD  463303 
and  AD  463304).  This  report  describes  the  reliability  program 
management  activities  of  the  SPO  as  related  to  the  Space  Systems 
Division,  AFSC;  and  responds  to  the  requirements  of  MIL-R- 
27542A(USAF) ,  "Reliability  Program  Requirements  for  Systems, 
Subsystems  and  Equipments."  It  does  not  contain  discussions  of 
reliability  engineering  techniques  and  procedures. 

Handbook  of  Reliability  Engineering.  NAVWEPS  00-65-502, 

1  June  1964.  This  handbook  presents  reliability  methods  for 
application  by  project  management  and  engineering  personnel 
within  the  Bureau  of  Naval  Weapons.  This  handbook  is  primarily 
concerned  with  engineering  practices  and  methods,  however,  and 
presents  management  concepts  in  a  cursory  manner. 

Reliability  and  Maintainability  Program  for  Material.  Combat 
Operations  Research  Group  Memorandum  CORG-M-181,  1  August 
1964.  (DDC  No.  AD474356).  This  document  presents  the  results 
of  a  study  to  define  the  reliability/maintainability  program  of  the 
U.  S.  Army  Combat  Development  Command,  based  on  interpreta¬ 
tion  of  AR  705-25  and  AR  705-26.  This  provides  interpretations 
of  Department  of  the  Army  policy,  and  should  be  used  with  care 
in  connection  with  Air  Force  programs. 
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Reliability  Design  Handbook,  NAVSHIPS  94501.  This  handbook 
describes  reliability  design  techniques  and  procedures  and  in¬ 
cludes  discussions  of  various  aspects  of  the  reliability  programs 
of  the  U.  S.  Navy  Ships  Systems  Command.  This  handbook  pro¬ 
vides  a  brief  discussion  of  certain  management  considerations, 
but  is  primarily  engineering  design  oriented.  The  specific  activi¬ 
ties  of  system  life  cycle  phases  are  not  defined. 
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CHAPTER  3 


RELIABILITY  PROGRAM  MANAGEMENT 

1  INTRODUCTION 

In  view  of  the  ciitical  need  for  effective  system  management  during 
the  entire  life  cycle  of  the  system,  the  Air  Force  System  Command 
has  instituted  a  System  Program  Management  Procedure  which  is 
described  in  detail  in  manual  AFSCM  375-4.  This  document  provides 
direction  and  guidance  for  management  of  a  phased  program  as  appli¬ 
cable  to  the  conception,  definition,  acquisition  and  operation  of  large- 
scale  systems.  However,  the  basic  concepts  of  AFSCM  375-4  are 
also  applicable  to  many  non-system  programs  with  management  re¬ 
quirements  similar  to  those  of  major  system  programs  Many  of  the 
program  functions  also  are  applicable  to  procurement  activities,  even 
though  specific  life  cycle  phases  may  not  be  destined. 

Throughout  the  system  life  cycle,  many  managerial  and  '■echnical  dis¬ 
ciplines  are  applied  to  assure  a  suitable  system  within  constraints  of 
various  parameters  such  as  cost  and  time.  Reliability  management 
and  engineering  activities  included  throughout  the  system  life  cycle 
are  included  among  the  more  significant  of  these  disciplines. 

Many  program  functions  defined  in  AFSCM  375-4  require  data  result¬ 
ing  from  specific  activities  of  a  reliability  program.  However,  with 
few  exceptions ,  these  activities  are  not  specifically  defined  in  relation 
to  reliability  program  requirements.  The  objective  of  this  chapter  is 
to  identify  specific  system  program  management  actions  involving  or 
related  to  reliability  program  activities,  to  briefly  describe  these 
activities  and  indicate  their  relationship  to  the  overall  system  program 
management  structure. 

2.  OBJECTIVES  O  F  RELIABILITY  PROGRAM  MANAGEMENT 

The  final  objective  of  syrstem  program  management  is  the  timely  de¬ 
livery  of  systems  meeting  defined  operational  requirements  within  the 
constraints  of  available  resources.  In  support  of  this,  the  final 
objective  of  a  reliability  program  is  to  assure  delivery  of  systems  that 
meet  specified  reliability  requirements. 

A  group  of  specific  objectives  of  reliability  program  management  can 
be  defined  which,  if  achieved,  will  help  to  assure  that  the  final  objec¬ 
tive  is  reached.  These  specific  objectives  are: 

a.  Provide  the  framework  for  assuring  appropriate  consideration  of 
reliability  requirements  in  establishing  functional  and  physical 
configuration  of  the  system. 
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b.  Insure  an  effective  reliability  program  during  system  definition, 
acquisition  and  operation. 

l  .  Balance  reliability  factors  against  other  factors  such  as  perform¬ 
ance,  lime  and  cost  to  obtain  the  required  system. 

d.  Minimize  the  technical,  economical  and  schedule  risks  in  assur¬ 
ing  reliability  achievements  and  verification  during  the  development 
and  production  effort 

e.  Control  reliability  aspects  of  changes  in  system  requirements 
during  development  and  production.  This  includes  changes  per¬ 
formed  to  achieve  a  specified  level  of  reliability,  as  well  as  those 
that  are  performed  for  other  purposes,  but  which  may  impact  on 

r  el  i  ability. 

L  Establish  a  high  probability  of  success  in  obtaining  a  reliable 
system  in  a  timely,  economical  manner. 

g.  Document  decisions  concerning,  and  impacting  on  the  reliability 
program. 

h.  Establish  a  discipline  for  the  reliability  elements  of  a  System 
Program  Office  (SPO)  to  follow  so  that  a  closed-loop  effort  is 
maintained  between  the  reliability  activities  and  other  associated 
activities  such  as  maintainability,  safety,  and  human  factors; 
and  with  the  functional  areas  of  procurement  and  production,  pro¬ 
gram  control,  configuration  management,  system  engineering, 
test  and  deployment,  and  logistics. 

i.  Manage  and  control  the  reliability  program  efforts  of  contractors. 
Identify  significant  reliability  program  functions  to  lie  performed 
by  other  organizations  such  as  Air  Force  Logistic  Command 
(AFLC)  and  using  commands  participating  in  systems  management. 

j.  Establish  requirements  for  flow  of  reliability  and  related  infor¬ 
mation  between  responsible  organizations. 

k.  Accomplish  or  manage  the  accomplishment  of  reliabilty  program 
actions  as  identified  for  the  definition,  acquisition  and  operational 
processes. 

3.  RELIABILITY  PROGRAM  MANAGEMENT  ACTIVITIES 

Reliability  program  management  activities  throughout  the  system  life 
cycle  are  discussed  in  the  following  paragraphs.  These  are  presented 
as  a  general  discussion  representing  a  "typical"  system  development 
program,  ard  follow  the  concepts  of  the  system  program  management 
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procedures  presented  in  AFSCM  375  -4.  Actual  reliability  program 
management  activities  performed  in  support  of  a  particular  system  de¬ 
velopment  program  could  vary  somewhat  depending  on  the  spefific  pro¬ 
gram  requirements  and  the  type  of  system  involved.  However,  the 
fundamentals  presented  here  are  applicable  to  any  program.  Key  reli- 
bility  program  activities  that  are  applicable  even  in  the  case  of  small- 
scale  development  or  procurement  programs  where  identified  phases 
of  a  system  development  program  are  difficult  or  impossible  to  define, 
are  emphasized  in  the  respective  discussions. 

The  system  life  cycle  phases,  as  defined  in  AFSCM  375-4,  and  the 
fundamental  purpose  of  each  of  the  phases  are  as  follows: 

Conceptual  Phase:  Develop  requirements  and  concepts  for  Air 
Force  systems  which  will  fulfill  military  defense  objectives. 

Definition  Phase:  Sufficiently  define  the  cost,  schedule,  and 
system  elements  required  to  satisfy  the  requirements  developed 
during  the  Conceptual  Phase. 

Acquisiton  Phase:  Acquire  and  test  the  system  elements  as  defined. 

Operational  Phase:  Provide  the  using  command  or  organization  with 
the  system  elements  and  the  logistics  and  engineering  support  re¬ 
quired  to  accomplish  the  mission  of  the  system. 

RELIABILITY  PROGRAM  MANAGEMENT  DURING  THE  CONCEPTUAL 
PHASE 

Reliability  program  management  activities  performed  during  the  Con¬ 
ceptual  Phase  should  be  directed  toward  establishing  appropriate  and 
feasible  system  reliability  objectives  or  goals.  During  early  stages  of 
this  phase,  the  foundation  is  established  for  specific  reliability  program 
activities  that  become  evident  later  in  the  phase.  Usually,  the  first 
input  identifiable  as  specific  activities  of  the  reliability  program  are 
those  concerned  with  the  quantification  of  reliability  requirements  in 
preparation  for  the  initiation  of  the  Definition  Phase.  However,  these 
require  nents  are  based  on  the  results  of  earlier  activities  during  sys¬ 
tem  planning  studies  or  exploratory  and  advanced  development.  There¬ 
fore,  early  Conceptual  Phase  activities  are  summarized  here  in  relation 
to  their  impact  on  later  reliability  program  activities. 

4.  1  Early  Conceptual  Phase  Activities 

The  Conceptual  Phase  includes  activities  directed  toward  identi¬ 
fication  and  formulation  of  sy’Hm  requirements,  development  of 
the  system  concept  (system  planning),  and  development  of  new 
technology.  These  three  areas  of  activity  progress  concurrently 
toward  the  Conceptual  Transition  activities  that  terminate  the  Con¬ 
ceptual  Phase  and  initiate  the  Definition  Phase. 
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The  first  specific  requirements  for  an  identifiable  system  are 
formulated  by  a  Using  Command  and  are  documented  in  a  Qualita¬ 
tive  Operational  Requirement  (QOR).  This  document  describes 
the  requirement  for  an  operational  capability,  describes  the 
threat  environment,  and  postulates  an  operational  concept,  and  con¬ 
tains  much  of  the  mission-oriented  information  that  will  be  used 
later  in  establishing  system  reliability  goals. 

The  first  major  involvement  of  AFSC  in  system  development  is 
the  long-range  system  planning  centered  about  the  AFSC  Techno¬ 
logical  War  Plan  (TWP).  This  plan  responds  to  the  QOR,  and 
defines  environmental,  technological,  and  resource  requirements 
of  the  proposed  system.  Therefore,  some  of  the  important  con¬ 
straints  that  will  be  imposed  on  the  reliability  development  activi¬ 
ties  are  identified  in  the  TWP. 

Where  the  need  is  evident,  the  TWP  initiates  Systems  Planning 
Studies  which  consider  qualitative  factors  relating  to  operational 
and  technical  capability,  together  with  the  defined  operational 
requirements  factors  in  more  completely  defining  the  require¬ 
ments  of  the  system.  Thus,  the  System  Planning  Studies  provide 
information  directly  related  tc  subsequent  development  of  system 
reliability  objectives. 

The  other  area  of  activity,  technological  development,  includes  the 
exploratory  and  advanced  development  activities  that  are  based  on 
findings  of  previous  research  programs  as  well  as  on  identified 
system  requirements.  In  many  cases,  advances  in  reliability- 
technology  are  obtained  as  a  direct  or  indirect  result  of  research, 
and  it  is  essential  that  such  knowledge  be  included  in  the  relia¬ 
bility  aspects  of  system  development  activities. 

Technological  development  activities,  identifiable  as  Exploratory 
and  Advanced  Development,  are  directed  toward  specific  military 
problem  areas,  and  development  of  advanced  technological  con¬ 
cepts  that  are  directly  applicable  to  specific  systems.  Such  de¬ 
velopment  efforts  normally  involve  investigation  and  development 
of  operational  or  performance  concepts.  However,  it  is  possible 
for  a  stated  level  of  achieved  reliability  to  be  a  primary  develop¬ 
ment  objective.  In  any  case,  the  results  of  these  development 
activities  of  the  early  Conceptual  Phase  should  be  given  careful 
consideratio  i  in  establishing  the  reliability  goals  for  the  system, 
and  in  developing  the  approach  for  meeting  these  goals. 

4.2  Reliability  Program  Activitie s  During  Conceptual  Transition 

Following  the  successful  completion  of  the  early  system  require¬ 
ment,  system  planning  and  technological  development  activities- 
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and  upon  receipt  oi  a  Specific  Operation  Requirement  (SOR), 
Operational  Support  Requirement  (OSR),  or  specified  Advanced 
Development  Objective  (ADO),  a  series  of  Conceptual  Transition 
activities  are  performed  to  prepare  for  the  initiation  of  the 
Definition  Phase.  Typically,  the  System  Program  Office  cadre 
is  established  at  this  time  and  the  first  actions  clearly  identifi¬ 
able  as  Reliability  Program  activities  are  initiated. 

The  most  significant  efforts  during  Conceptual  Transition  are  the 
System  Engineering  activities  whereby  operational  requirements 
are  translated  into  system  performance  requirements.  It  is 
during  this  effort  that  requirements  for  system  effectiveness  are 
first  interpreted  in  terms  of  reliability,  human  performance, 
safety  and  maintainability.  The  results  of  the  Conceptual  Transi¬ 
tion  engineering  effort  provide  a  significant  input  to  the  Preliminary 
Technical  Development  Plan  which,  together  with  the  Program 
Change  Proposal  (PCP),  the  Military  Construction  Program 
(MOP),  and  the  Secretary  of  the  Air  Force's  Determinations  and 
Findings  (D&F)  forms  the  Program  Requirements  Baseline,  which 
governs  the  activities  of  the  Definition  Phase. 

The  reliability  program  management  efforts  during  Conceptual 
Transition  should  be  directed  toward  assuring  the  accomplishment 
of  a  variety  of  related  activities  such  as  those  described  below. 
(Each  of  these  areas  of  activity  are  discussed  more  thoroughly 
in  the  chapters  referenced  in  parentheses. ) 

a.  Reviewing  previous  documents  to  identify  all  factors  pertinent 
to  the  reliability  program,  and  maintaining  an  updated  listing 
of  all  such  documentation.  (Chapter  4) 

b.  Quantifying  the  gross  level  of  reliability  which  must  be  met 
to  satisfy  system  requirements.  (Chapters  4  and  6) 

c.  Developing  reliability  block  diagrams  reflecting  the  functional 
relationships  of  the  system.  (Chapters  8,  9,  and  11) 

d.  Identifying  system  functions  of  particular  importance  to  the 
reliability  development  effort.  This  includes  functions  de¬ 
fining  operating  periods,  cycles,  or  major  mission  segments 
nn  which  reliability  requirements  are  to  be  based,  and 
additional  functions  defining  the  constraints  affecting  relia¬ 
bility  achievements.  (Cha.pters  4,  6,  8,  and  11) 

e.  Performing  basic  reliability  allocation  studies.  (Chapter  8) 
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i.  Preparing  the  reliability  program  requirements  input  for  the 
initial  PI  DP.  This  includes  development  of  such  factors  as: 

Reliability  apportionment,  prediction,  and  modeling. 

(Chapters  8  and  9) 

Expected  environmental  conditions.  (Chapter  ]i) 

Requirements  for  reliability  participation  in  Design 

Reviews.  (Chapters  4  and  6) 

Requirements  for  reliability  tests,  demonstration  and 

resolution  of  problem  areas.  (Chapters  6  and  10) 

4.  3  Initial  Reliability  Program  Activities  in  Non-System  Procurements 

During  the  early  stages  or  non-system  programs  for  the  procure¬ 
ment  of  individual  equipment  items,  experimental  models,  com¬ 
mercial  items,  and  similar  procurements,  a  conceptual  phase, 
as  such  is  seldom  defined.  However,  certain  reliability  pregram 
activities  are  necessary  for  most  programs  such  as  those  starting 
at  the  time  that  the  operational  requirements  and  design  goals 
are  being  defined,  and  continuing  throughout  the  design  and  pro¬ 
duction  stages  of  the  procurement  program.  The  earliest  of  these 
activities  would  be  performed  at  a  time  during  the  equipment 
development  cycle  that  was  equivalent  to  the  conceptual  phase 
of  the  system  life  cycle. 

In  certain  programs,  such  as  in  the  development  and  procurement 
of  large  equipments  for  the  normal  inventory,  all  reliability  pro¬ 
gram  activities  indicated  in  paragraph  4.  2  may  be  necessary.  In 
other  cases,  the  reliability  program  should  be  scaled  down  to  be 
commensurate  with  technical  and  economical  constraints  of  the 
particular  procurement.  As  a  minimum,  however,  the  reliability 
program  should  include  the  activities  necessary  for  reviewing 
related  documentation,  developing  gross  reliability  requirements, 
and  defining  the  reliability  program  activities  to  be  performed 
during  design  and  production.  (See  items  a,  b  and  f  of  paragraph 
4.  2.  Also  see  paragraph  5  of  Chapter  5.  ) 

5.  RELIABILITY  PROGRAM  MANAGEMENT  DURING  THE  DEFINITION 
PHASE 

In  general,  the  Definition  Phase  is  divided  into  three  subphases  as 
follows : 

Phase  A  includes  the  activities  necessary  to  establish  the  formal 
SPO  and  prepare  for  contractor  definition. 
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Pha.;e  B  includes  the  efforts  of  competing  contractors  in  perform¬ 
ing  tne  definition  tasks,  and  in  preparing  their  proposals. 

Phase  C  includes  the  Air  Force  efforts  in  evaluating  proposals 
and  selecting  the  development  contractor. 

Reliability  program  management  activities  performed  during  the  Defini¬ 
tion  Phase  should  be  directed  toward  defining  the  cost,  schedule,  and 
technical  design  approach  to  satisfy  the  system  reliability  requirements. 

In  general,  these  activities  will  include  efforts  such  as  preparation  of 
reliability  requirements  for  the  system  specification,  preparation  of 
reliability  program  plans,  determining  realistic  cost  and  schedule  esti¬ 
mates  for  reliability  engineering  in  relation  to  other  engineering,  logistic 
production  and  support  cost,  and  identifying  high  risk  areas.  In  addition, 
the  system  reliability  program  is  interpreted  in  terms  of  subsystem 
factors,  firm  and  achievable  reliability  requirements  are  allocated  to 
subsystems  and  reliability  requirements  are  evaluated  with  reference 
to  contracting  for  the  system  development  activities  of  the  Acquisition 
Phase. 

Reliability  program  activities  performed  during  the  Definition  Phase 
of  a  typical  system  development  program  are  described  below.  In  addi¬ 
tion,  sequential  diagrams  of  reliability  program  activities  during  Phase  A 
and  Phase  B  are  shown  in  Figures  3-1  and  3-?.,  respectively. 

The  discussion  of  reliability  program  activities  during  the  Definition 
Phase  is  presented  with  reference  to  a  system  development  program. 
However,  many  of  the  activities  discussed  in  paragraph  5.  1  through 
5.  3  below  are  also  applicable  to  ncn-system  programs.  This  is  espe¬ 
cially  true  during  the  normal  procurement  of  new  equipment  items 
for  the  operational  inventory  when  the  development  program  includes 
activities  equivalent  to  the  Definition  Phase  of  a  system  development 
program.  Those  activities  that  will  be  most  important  to  a  program  of 
this  type  are: 

.  Preparing  reliability  input  for  contractor  definition  SOW.  (See 
paragraph  5.  1  f.  ) 

.  Evaluating  reliability  factors  in  contractor  definition  proposals. 

(See  paragraph  5.  1  i.  ) 

.  Preparing  reliability  requirements  input  for  CEI  Detail  Specifi¬ 
cations.  (See  paragraph  5.  2d.) 

.  Preparing  Initial  Reliability  Test  Plans.  (See  paragraph  5.2e.) 

.  Evaluating  contractors  proposals  for  development.  (See  para¬ 
graph  5.3.) 
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5.  1  Reliability  Program  Activities  During  Phase  A 

Following  the  receipt  of  the  System  Definition  Directive  (SDD),  SPO 
cadre  is  augmented  to  create  the  formal  System  Program  Office 
(SPO)  under  the  direction  of  the  System  Program  Director.  The 
SPO  is  composed  of  two  divisions  performing  the  staff  functions  of 
Program  Control  and  Configuration  Management,  and  three  line 
divisions  headed  by  the  Deputy  Director  for  Procurement  and  Pro¬ 
duction,  the  Deputy  Director  for  Engineering,  and  the  Deputy 
Director  for  Test  and  Deployment. 

The  responsibility  for  reliability  engineering  during  Phase  A  is 
delegated  to  the  Deputy  Director  for  Engineering  who  assigns  relia¬ 
bility  specialists  to  emphasize  the  reliability  discipline  as  an 
integral  part  of  the  total  system  engineering  process.  In  addition 
to  reliability  engineering  activities,  however,  the  reliability  pro¬ 
gram  also  supports  the  program  control,  configuration  control, 
procurement,  and  test  activites  of  the  other  divisions.  This  wide 
range  of  program  support  activities  during  Phase  A  is  illustrated 
in  Figure  3-1.  The  reliability  program  activity  blocks  in  this  dia¬ 
gram  are  shown  in  rows  corresponding  to  the  most  closely  associ¬ 
ated  area  of  system  management.  In  addition,  each  block  of 
Figure  3-1  is  keyed  to  one  or  more  blocks  of  Figure  6  of  AFSCM 
375-4.  The  reliability  program  activities  are  typically  performed 
in  the  sequence  as  indicated  by  the  diagram.  However,  in  a  particu¬ 
lar  program,  the  sequence  of  activities  may  vary,  or  several  activ¬ 
ities  may  be  combined  or  performed  concurrently.  The  reliability 
program  activities  indicated  by  each  block  are  described  more  fully 
in  the  following  discussions  which  are  identified  according  to  the 
respective  block  of  Figure  3-1.  The  numbers  in  parentheses  follow¬ 
ing  the  subject  headings  refer  to  blocks  in  Figure  6  of  AFSCM  375-4. 

a.  Review  and  Update  Reliability  Input  to  Program  Requirements 
Baseline  (1,2  and  3).  One  of  the  initial  activities  of  the  SPO  is 
the  review  and  revision  of  the  PTDP  with  respect  to  other  re¬ 
quirements  of  the  System  Definition  Direction.  At  this  time, 
control  of  technical  inputs  to  the  Program  Requirement  Base¬ 
line  (i.  e.  ,  the  performance  requirements,  design  criteria, 
and  other  data  defining  the  technical  requirements  of  the  system) 
is  assumed  by  the  Configuration  Management  Division.  Direct 
support  is  provided  by  the  Deputy  Director  for  Engineering, 
who  is  responsible  for  the  development,  integration,  interface 
compatability ,  and  validity  of  the  reliability  input.  Thus,  the 
earliest  activities  of  the  reliability  specialists  should  include 
reviewing  the  reliability  and  related  requirements  and  develop¬ 
ing  recommended  changes  to  the  PTDP.  Typically,  such  recom¬ 
mendations  would  be  approved  by  the  Deputy  Director  for 
Engineering  before  submission  to  the  Configuration  Management 
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Division  for  final  approval.  The  approved  change  is  then 
incorporated  in  tne  PTDP  by  the  SPO  Program  Control 
Division. 

b .  Define  Reliability  Design  and  Trade-Off  Study  Requirements 

(4,  5  and  6).  The  reliability  engineering  process  started  during 
Conceptual  Transition  is  continued  as  the  first  major  reliability 
engineering  efforts  of  the  Definition  Phase.  The  system  relia¬ 
bility  requirements  are  expanded  to  establish  a  basis  for  allo¬ 
cation  of  various  requirements  among  the  system  elements  and 
defining  subsystem  reliability  requirements  as  required  for 
preparing  the  initial  System  Specification.  (See  Chapter  8  for 
discussion  of  Reliability  Allocation  Procedures.) 

In  the  process  of  determining  system  performance  and  design 
requirements,  and  during  the  process  of  defining  the  various 
requirements  and  constraints  such  as  safety,  reliability,  and 
maintainability,  many  alternative  methods  will  be  identified, 
the  most  fruitful  of  which  should  be  selected  for  input  to  trade¬ 
off  studies.  At  this  time,  the  reliability  considerations  should 
be  examined  from  a  total  systems  point  of  view  to  identify  sub¬ 
sequent  trade-off  study  considerations.  An  important  function 
of  system  reliability  engineering  at  this  time  is  that  of  assuring 
that  reliability  will  be  given  its  appropriate  weight  and  that 
essential  constraints  are  established  such  that  the  achievable 
level  of  system  reliability  is  not  degraded  beyond  acceptable 
limits.  Trade-off  studies  involve  the  application  of  many  dis¬ 
ciplines  as  discussed  in  this  notebook.  The  effect  of  trade-offs 
on  system  reliability  are  usually  evaluated  using  reliability  pre¬ 
diction  procedures  as  described  in  Chapter  9- 


c .  Prepare  Reliability  Requirements  for  System  Specification 
(7  and  8).  The  first  major  configuration  control  activity 
following  the  assumption  of  technical  control  of  the  baseline 
documents  by  the  SPO,  is  that  of  preparing  the  initial  System 
Specification  to  be  included  in  the  Phase  B  statement  of  work. 

The  management  control  of  the  System  Specification  is  the 
re sponsibility  of  the  Configuation  Management  Division.  How¬ 
ever,  all  technical  input,  including  the  reliability  requirements 
arc  the  responsibility  of  the  Deputy  Director  for  Engineering. 

The  System  Specification  includes  a.  paragraph  that  specifies 
the  system  reliability  requirements  in  quantitative  terms.  In 
addition,  the  System  Specification  states  system  reliability 
acceptance  testing  requirements  ^see  Exhibit  I  of  AHSCM 
375-1).  Development  of  these  requirements  should  be  the 
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direct  responsibility  of  the  reliability  spe  ialist,  who  should 
be  supported  by  the  Test  and  Deployment  Division  in  the 
development  of  reliability  test  requirements. 

Techniques  used  in  the  development  of  reliability  requirements 
are  discussed  in  the  reliability  specification  portion  of 
Chapter  6. 

Prepare  Initial  Test  Program  Plans  (9).  The  Deputy  Director 
for  Test  and  Deployment  is  responsible  for  initiating  the 
development  of  plans  for  the  su  ^vcnt  testing  of  the  system. 
These  are  essentially  management-oriented  planning  docu¬ 
ments  to  guide  the  accomplishment  of  the  overall  system  test 
program,  assure  adequate  lead  time  for  development  of  test 
procedures  and  facilities,  and  provide  a  basis  for  more  de¬ 
tailed  planning  and  operating  documents.  Any  requirements 
for  extensive  and  time  consuming  reliability  testing  are 
important  inputs  in  the  development  of  these  plans,  and  identi¬ 
fication  of  any  such  requirements  should  be  initiated  at  this 
time.  Therefore,  reciprocal  support  will  be  essential  between 
the  reliability  and  test  specialists  in  developing  reliability 
test  requirements  for  the  System  Specifications,  and  in  pre¬ 
paring  initial  reliability  test  program  plans. 

Some  of  the  factors  to  be  considered  in  the  development  of 
reliability  test  program  plans  are  discussed  in  Chapter  6. 

Also  details  of  the  development  of  reliability  test  procedures 
are  presented  in  Chapter  10. 

Define  Reliability  Program  Milestone  (10,  11  and  1Z).  The 
first  major  activity  of  the  Program  Control  Division  of  the 
SPO,  following  the  verification  of  the  Program  Requirements 
Baseline,  is  that  of  developing  the  preliminary  Program  Work 
Breakdown  Structure  (PBS)  and  Program  Management  Network 
(PMN).  This  function  is  primarily  system  program  manage¬ 
ment  oriented,  but  is  influenced  by  systems  engineering  and 
other  groups  to  the  extent  necessary  for  assuring  the  defini¬ 
tion  of  all  critical  milestones  of  the  network.  In  particular, 
the  system  reliability  management  inputs  should  include  re¬ 
quirements  and  schedules  for  development  of  reliability  pro¬ 
gram  plans,  reliability  design  reviews,  reliability  test  plans, 
and  other  reliability  milestones  of  the  overall  PMN. 

Specific  activities  and  milestones  of  the  reliability  program 
are  summarized  in  this  chapter.  In  addition,  the  reliability 
engineering  management  activities,  and  reliability  data  man¬ 
agement  considerations  as  discussed  in  Chapters  4  and  5, 
respectively,  will  aid  in  identifying  the  reliability  program 
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activities,  and  establishing  associated  milestones  for  the 
Program  Management  Network. 

f .  Prepar  e  Re  liability  Input  for  Phase  B  Statcmen  t  of  Work 
(14).  The  Deputy  Director  for  Engineering  is  responsible 
for  preparation  of  the  Statement  of  Work  (SOW)  for  the 
Phase  B  effort.  The  most  significant  portion  of  the  Phase  B 
SOW,  insofar  as  reliability  program  management  is  con¬ 
cerned,  is  the  statement  of  Phase  B  system  engineering 
effort  which  includes  requirements  for  trade-off  studies 
involving  system  reliability  as  one  of  the  principal  param¬ 
eters  of  system  effectiveness.  -  Also,  specific  reliability 
requirements  should  be  stated  in  the  requirements  for  pre¬ 
paring  the  Part  I  Detail  Specifications  for  each  CEI,  and  in 
updating  the  System  Specification. 

Other  portions  of  the  SOW  that  include,  or  are  impacted  by 
reliability  considerations  are  the  requirements  for  system 
reliability  evaluation  in  design  reviews,  and  the  requirements 
for  development  of  reliability  program  management  plans. 

The  reliability  specialist  should  also  review  documents 
referred  in  the  SOW  to  assure  that  appropirate  reliability 
specifications  are  imposed,  'chat  only  essential  reliability 
requirements  are  listed  as  applicable,  and  that  duplicate  or 
contradictory  requirements  are  not  generated  by  secondary 
reference.  It  is  the  reliability  specialist's  responsibility  to 
determine  the  reliability  data  items  to  be  specified  in  the 
Contract  Data  Requirements  List,  DD  form  14Z3.  See 
Chapter  5  for  guidance  in  establishing  reliability  program 
data  requirements. 

g .  Providing  Reliability  Program  Assistance  in  Phase  A  Pro- 
curement.  Activities  (19  and  Zl).  A  series  of  procurement 

and  production  management  actions  are  performed  to  complete  1 1, « 
request  for  proposals  (RFP),  and  solicit  bids  for  performing 
Phase  B.  These  actions  involve  activities  of  all  participants, 
including  the  reliability  specialists  from  the  office  of  the 
Deputy  Director  for  Engineering.  Some  of  the  more  signifi¬ 
cant  activities  of  the  reliability  specialists  during  the  finaL 
procurement  actions  include:  1 

.  Updating  the  reliability  inputs  to  the  Phase  B  SOW. 

.  Performing  final  review  and  verification  of  the  reliability 
requirements  in  the  intial  System  Specification. 

.  Reviewing  and  updating  as  necessary  the  system  relia¬ 
bility  test  plan. 
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.  Developing  criteria  for  evaluating  the  reliability  factors 
in  proposals. 

.  Review  information  relating  to  any  reliability  achieve¬ 
ment  incentive  provisions  in  the  RFP. 

.  Providing  consultation  or  direct  assis  ance  to  the  Source 
Selection  Advisory  Council  (SSAC)  in  rating  reliability 
provisions  in  contractors'  proposals. 

.  Providing  assistance  to  pre -proposal  briefings  where 
questions  maybe  asked  concerning  technical  aspects  of 
the  reliability  program. 

h.  Contractor  Reliability  Program  Efforts  in  Proposal  Prep¬ 
aration  (22  through  27).  Contractors  selected  to  submit  pro¬ 
posals  will  typically  perform  a  series  of  iterative  actions 
that  culminate  with  a  definitive  proposal  for  performing 
Phase  B.  One  of  the  major  activities  in  proposal  preparations 
is  in  performing  selected  studies  and  identifying  additional 
trade-off  study  requirements  for  Phase  B.  One  of  the  major 
efforts  in  the  support  of  such  trade-off  studies  will  be  the 
expansion  of  the  reliability  model,  and  refinement  of  the 
reliability  allocations  to  reflect  the  contractor's  proposed 
system,  design  characteristics. 

The  contractors'  proposal  development  should  also  include 
updating  and  verifying  the  reliability  requirements  provisions 
in  the  System  Specification. 

Ideally,  each  contractor  will  review,  verily  and  expand  the 
reliability  requirements  paragraph  of  the  System  Specification 
based  on  the  expanded  reliability  model  and  refined  reliability 
allocations  to  provide  reliability  requirements  for  identifiable 
system  elements.  Additional  activities  performed  by  the  con¬ 
tractor  should  include  verifying  and  expanding  the  reliability 
program  activities  and  milestones  of  the  PMN. 

i.  Evaluate  Reliability  Factors  in  Phase  B  Proposals  (33).  Pro¬ 
posals  received  from  the  several  contractors  in  response  to 
the  Phase  B  RFP  are  evaluated  using  previously  established 
evaluation  criteria.  Each  technical  and  managerial  factor 

of  the  proposal  is  scored  according  to  an  objective  scoring 
system,  and  the  contract  is  awarded  accordingly. 

The  SPO  reliability  program  should  include  provisions  for 
evaluating  the  reliability  aspects  of  the  proposals.  The  weight 
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accorded  reliability  factors  daring;  the  proposal  evaluation 
will  vary  depending  on  the  requirements  of  the  system  under¬ 
development.  This  weight  may  or  mav  not  constitute  a 
signiiic.ini  portion  of  the  total  score.  However, 
regardless  of  die  relative  weights  established  for  the  relia¬ 
bility  factors,  failure  to  meet  the  minimum  reliability  re 
quirements  can  be  grounds  for  rejecting  the  proposal,  even 
though  all  other  factors  meet  the  SOW  requirements. 

5.2  Reliability  Program  Activities  During  Phase  B 

Following  final  review,  the  Phase  B  contracts  are  signed  and  dis¬ 
tributed  to  tlie  contractors  to  officially  initiate  Phase  B.  From 
this  point  until  the  completion  of  Phase  B  the  SPO  will  fully  support 
and  coordinate  the  contract')',  effect.  The  contractor's  activites 
during  Phase  B  represent  an  iteration  in  depth  of  the  preceding 
activities,  including  the  preparation  of  the  complete  System  Speci¬ 
fication,  and  preparation  o:  detailed  plans  and  schedules  for  system 
development.  Figure  3-2  indicates  the  areas  in  which  the  contractu 
should  provide  for  significant  reliability  program  activites  in  a 
typical  system  definition  program,  and  to  which  the  SPO  should  give 
particular  attention  in  monitoring  and  coordinating  the  contractors 
reliability  program  activities. 

The  activity  blocks  ir.  Inis  diagram  are  shown  in  three  rows  to 
indicate  the  SPO  management  areas  most  directly  associated  with 
ihe  respective  reliability  program  activities  performed  by  the  con¬ 
tractor.  In  addit.on,  each  block  of  Figure  3-2  is  keyed  to  one  or 
more  blocks  ox  Figure  6  of  AFSCM  375-4. 

The  contractor's  reliability  program  typically  includes  those  activi 
ties  indicated  in  the  diagram.  However,  since  each  contractor  will 
organize  his  reliability  program  to  conform  to  his  overall  manage¬ 
ment  structure,  t.he  sequence  of  activities,  and  even  the  relative 
emphasis  placed  on  each  of  the  various  activities  will  vary  from 
contractor  to  contractor. 

The  reliability  program  activities  indicated  in  Figure  3  2  are  dis¬ 
cussed  more  fully  in  the  following  paragraphs.  The  numbers  in 
parentheses  following  the  subject  headings  refer  to  the  blocks  in 
Figure  6  of  .  cFSCM  375-4. 

a .  Perform  Reliability  Engineering  and  Analysis  for  Trade-off 

Studies  (42).  A  major  effort  of  Phase  B  is  the  performance  of 
trade-of:  studies  to  assure  the  best  possible  balance  among 
total  cost,  schedules  and  operational  effectiveness  factors. 

The  effect  of  reliability  factors  should  be  considered  during 
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all  trade-off  studies,  not  only  in  relation  to  system  effective¬ 
ness  achievement,  but  also  in  relation  to  other,  often  more 
obscure  factors  such  as  maintenance  support  cost.  For  ex¬ 
ample,  logistics  factors  such  as  maintenance  spare  provision- 
ing  plans  are  directly  influenced  by  component  failure  rate. 

Typical  reliability  engineering  and  analysis  activites  associated 
with  virtually  all  trade-off  studies  include  such  diversified 
tasks  as: 

.  Updating  the  system  reliability  model  to  assess  the  relia¬ 
bility  characteristics  of  alternative  design  approaches. 

.  Providing  failure  rate  prediction  data  for  alternate 

approaches  for  application  to  studies  such  as  logistics 
cost  vs.  initial  costs. 

.  Evaluating  historical  reliability  data  as  applicable  for 

activities  such  as  assessing  caveat  equipment  in  the  DOD 
inventory  in  selecting  alternate  «  : El 1  s . 

b.  Update  Reliability  Requirements  of  S  stem  Specifications  (44) . 
The  results  of  the  contractors  system  engineering  and  trade¬ 
off  study  effort  are  used  to  update  and  refine  the  System  Speci¬ 
fication.  During  this  updating,  particular  attention  is  paid  to 
the  reliability  provisions,  where  a  valid  allocation  of  reliability 
requirements  to  the  subsystems  is  a  prerequisite  to  the  sub  - 
sequent  development  of  Part  I  Detail  Specifications  for  the 
contract  end  items  program.  Therefore,  the  contractor's 
configuration  control  program  should  be  fully  supported  by 
reliability  engineering  in  assuring  that  system  reliability  re¬ 
quirements  are  properly  specified. 

c.  Develop  Reliability  Requirements  for  CEI's  (43,45  and  53). 

The  most  significant  activities  performed  by  the  contractor 
during  Phase  B  are  the  engineering  and  analysis  activities 
required  to  convert  the  gross  system  requirements  into  de¬ 
tailed  design  requirements  for  individual  CEI's.  This  includes 
the  development  of  all  design  requirements,  including  relia¬ 
bility  requirements. 

The  following  are  some  of  the.  more  important  reliability 
engineering  activities  associated  with  updating  the  reliability 
requirements  of  the  System  Specification: 

.  Reviewing  updated  baseline  data,  such  as  the  PTDP  and 
System  Specification  to  ensure  complete  understanding  of 
gross  system  reliability  requirements  and  constraints. 
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.  Updating  the  system  reliability  model  to  reflect  the 
system  configuration,  to  the  level  of  identified  CEI's. 

.  Reviewing  and  updating  reliability  allocations  to  develop 
specific  allocations  of  system  reliability  to  definable 
CEI’s. 

.  Establishing  quantified  reliability  requirements  for  the 
system,  subsystems,  and  definable  CEI's  and  assuring 
the  validity  and  practicality  of  these  requirements. 

.  Establishing  system  reliability  testing  requirements  com¬ 
patible  with  the  quantified  reliability  requirements,  and 
applicable  to  the  subsequent  preparation  of  reliability  test 
plans. 

d.  Prepare  Reliability  Requirements  Inputs  for  CEI  Detail  Speci¬ 
fications  (46  and  54).  Based  on  the  contractor's  development 
of  CEI  design  requirements,  the  Part  I  Detail  Specifications 
are  prepared  in  preliminary  form  for  each  identifiable  CEI. 

This  includes  the  preparation  of  the  requirements  and  test 
sections  (sections  3  and  4)of  the  specifications,  and  reflects 
the  information  in  the  updated  System  Specification  and  inte¬ 
grated  system  test  requirements.  These  specifications  include 
quantified  reliability  requirements  for  each  CEI.  Therefore, 
in  preparing  the  reliability  requirements,  close  cooperation 
will  be  required  between  system  engineering  and  configuration 
control  activities  to  assure  that  all  reliability  requirments 
and  test  provisions  are  carefully  prepared  and  adequate.  Some 
of  the  more  important  reliability  engineering  activities  that 
should  be  performed  in  preparing  the  reliability  inputs  for  the 
Part  I  Detail  Specifications  include: 

.  Reviewing  the  System  Specification  reliabilty  allocations 
with  respect  to  proposed  CEI  characteristics  to  verify 
the  validity  and  practicality  of  each  CEI  reliability  require¬ 
ment, 

.  Establishing  and  verifying  quantified  reliability  require¬ 
ments  for  inclusion  in  the  Part  I  Detail  Specifications. 

.  Establishing  CEI  reliability  testing  requirements  compati¬ 
ble  with  the  quantified  reliability  requirements,  and  appli¬ 
cable  to  preparation  of  category  I  test  plans. 


3-17 


e.  Prepare  Initial  Re-liability  Test  Plans  (47).  The  initial  test 
plans  are  refined  and  updated  by  the  contractor  in  the  prep¬ 
aration  of  initial  category  I  test  plans  and  inputs  for  subse¬ 
quent  category  II  test  plans.  Included  in  Lhese  plans  are 
tests  required  to  demonstrate  reli,  iity  achievement  for  the 
CEI's  during  category  I  testing,  and  for  the  integrated  sys'em 
during  category  II  testing. 

1.  Prepare  Reliability  Inputs  for  Phase  B  Final  Report  (60). 

(See  also  paragraph  31  of  AFSCM  375-4.  )  Phase  B  cf  the 
Definition  Phase  is  concluded  with  the  contractor's  sub¬ 
mission  of  his  Final  Report.  This  report,  which  includes 
the  contractor's  firm  proposal  for  development,  requires 
reliability  program  inputs  in  several  areas,  including: 

.  Reliability  aspects  of  trade-olf  conclusions. 

.  Reliability  inputs  to  system  engineering  documentation 
developed  during  Phase  B.  (See  Chapter  5.) 

.  Reliability  requirements  in  System  Specifica.tions  and  CEI 
Part  I  Detail  Specifications. 

.  Reliability  data  requirements  list  lor  the  development 
program  of  the  Acquisition  Phase.  (See  Chapter  5.  ) 

.  The  contractor's  reliability  program  management  plan. 

.  Identification  of  reliability  program  high  risk  areas. 

.  Identification  of  reliability  problems  that  could  not  be 

resolved  during  the  Definition  Phase. 

.  Reliability  program  activities  that  will  require  long  lead 
time  s . 

5.  3  Reliability  Program  Activities  During  Phase  C 

The  objective  of  Phase  C  is  to  select  the  definition  contractor  who 
is  to  continue  the  development  program  of  the  Acquisition  Phase. 
Therefore,  the  primary  activity  during  Phase  C  is  the  evaluation 
of  the  Phase  B  Final  Reports,  which  contain  the  contractor's 
firm  proposals  for  development,  and  the  selection  of  the  particular 
contractor  for  the  Acquisition  Phase  development  effort. 

The  evaluation  of  the  contractor's  final  report  typically  requires 
a  technical  evaluation  of  the  contractor's  design.  Such  evaluation 
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should  include  a  reliability  engineering  analysis  of  the  contractor's 
design  approaches  for  meeting  the  system  and  CEI  reliability 
requirements,  and,  as  such,  would  involve,  system  reliability 
modeling,  allocation  and  prediction  activities.  Thus,  reliability 
analysis  and  evaluation  support  will  be  essential  in  evaluating  fhe 
Phase  B  Final  Reports  and  proposals. 

After  being  updated  as  required  to  reflect  any  negotiated  changes, 
the  selected  contractor's  Phase  B  final  report  is  used  to  update 
and  refine  the  baseline  documents  that  will  govern  the  Acquisition 
Phase  development  effort.  Two  baselines  are  established  as 
follows : 

.  Design  Requirements  Baseline,  which  governs  the  configura¬ 
tion  management  of  the  system  development.  This  is  based 
on  the  Part  I  Detail  Specifications,  and  defines  all  design 
requirements,  including  system  and  subsystem  reliability 
requirements . 

•  Program  Requirements  Baseline,  which  governs  the  program 
management  efforts  of  the  acquisition  phase.  The  previous 
baseline  documentation  is  consolidated  to  provide  the  Proposed 
System  Packaging  Plan  (PSPP)  which  includes  recommendations 
for  full-scale  development. 

The  Definition  Phase  is  completed,  and  the  Acquisition  Phase 
initiated  with  the  issue  of  the  System  Program  Directive  (SPD)  by 
Hq.  USAF,  which  documents  official  approval  of  the  design  and 
program  requirement  baselines. 

6.  RELIABILITY  PROGRAM  MANAGEMENT  DURING  THE  ACQUISITION 
PHASE 

Reliability  program  management  activitie  s  performed  during  the  Acqui¬ 
sition  Phase  should  be  directed  toward  assuring  that  system  elements 
as  acquired  meet  the  reliability  requirements  of  the  System  Specifica¬ 
tion,  In  general,  these  activities  will  include  contractor  efforts  such 
as  detailed  reliability  engineering  activities,  updating  reliability 
factor  input  for  design  reviews,  planning  and  performing  relia¬ 
bility  demonstration  tests  as  part  of  categories  I  and  II  testing,  and 
providing  reliability  program  guidance  for  transition  to  the  Operational 
Phase.  SPO  activities  will  include  updating  baselines,  and  direction 
and  concurrent  support  of  testing  and  design  review  activities  through¬ 
out  the  Acquisition  Phase. 

A  sequential  diagram  indicating  reliability  program  activities  of  the 
Acquisition  Phase  is  shown  in  Figure  3-3.  The  activities  indicated  in 


this  diagram  are  related  to  general  areas  of  SPO  management  activity, 
and  keyed  to  corresponding  blocks  of  Figure  7  of  AFSCM.  375-4.  The 
reliability  program  activities  indicated  in  Figure  3-3  ar<  described 
in  the  following  discussion  of  Acquisition  Phase  activities.  The  numbers 
in  parentheses  following  each  subject  heading  also  refer  to  specific 
blocks  in  Figure  7  of  AFSCM  375-4. 

Several  of  the  activities  summarized  below  for  system  development 
during  the  Acquisition  Phase  are  also  applicable  to  non-system  pro¬ 
grams.  The  actual  activities  that  should  be  included  in  such  a  pro¬ 
gram  should  be  selected  based  on  the  requirements  of  the  specific 
program.  However,  any  development  program  should  include  relia¬ 
bility  engineering  support  activities  such  as  those  discussed  in  para¬ 
graphs  6c  and  6f.  In  addition,  support  should  be  provided  in  developing 
reliability  test  plans  and  in  performing  reliability  demonstration  test¬ 
ing  (see  paragraphs  6b  and  6g). 

a.  Update  Reliability  Factors  of  Design  Requirements  Baseline  (4). 

The  most  significant  activities  in  initiating  the  Acquisition  Phase 
insofar  as  the  SPO  reliability  program  is  concerned,  are  those 
involving  the  updating  of  the  Design  Requirements  Baseline  to 
reflect  changes  required  by  the  System  Program  Directive.  Partic¬ 
ular  emphasis  should  be  placed  on  the  requirements  for  compati¬ 
bility  between  the  Part  I  Detail  Specifications  and  the  System 
Specification.  Review  and  updating  of  reliability  requirements  of 
the  Design  Requirements  Baseline  is  a  system  engineering  activity. 
However,  these  activities  are  performed  in  support  of,  and  are 
directly  controlled  by  configuration  management. 

b .  Reliability  Program  Support  in  Developing  Category  1  and  II  Test 
Plans  (5  and  8).  The  responsibility  for  Category  I  and  II  testing 
is  assigned  to  the  SPO  Deputy  Director  for  Test  and  Deployment 
who  appoints  Air  Force  test  directors  who  will  be  responsible  for 
the  coordinated  development  of  category  I  and  II  test  plans,  and 
the  organization  of  a  field  test  force  for  conducting  category  II 
testing. 

Reliability  testing  is  a  major  factor  in  the  category  I  and  II  test 
programs.  In  fact,  the  importance  ol  reliability  testing  is  empha¬ 
sized  by  the  separate  paragraph.  ,pecifically  covering  reliability 
testing  in  both  the  System  Specification  and  the  Part  I  Detail 
Specifications  (see  Exhibits  I  and  II  of  AFSCM  375-1).  Therefore, 
reliability  testing  requirements  should  be  a  major  factor  in  the 
subsequent  test  program  planning  activities.  In  many  instances 
the  scope  of  reliability  testing  requirements  will  justify  the  appoint¬ 
ment  of  reliability  specialists  on  the  field  test  force  and,  for  very 
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large  systems  for  which  reliability  achievement  is  '  major  develop¬ 
ment  criterion,  a  reliability  specialist  could  be  in’.uded  on  the 
immediate  staff  of  the  Deputy  Director  for  Test  ana  Deployment. 

In  any  event,  the  reliability  specialists  of  the  System  Engineering 
Division  should  fully  support  the  Deputy  Director  for  Test  and 
Deployment  by  providing  technical  information  concerning  relia¬ 
bility  testing  and  test  facility  requirements. 

Preliminary  Reliability  Engineering  by  Contractor  (11).  One  of 
the  development  contractor's  early  activities  is  to  evaluate  and 
update  the  reliability  program  plans  and  schedules  which  were 
included  in  his  Phase  B  final  report,  and  prepare  his  working 
plans  for  providing  reliability  capabilities  for  the  development 
activities. 

The  development  contractor's  reliability  engineering  effort  begins 
as  a  continuation  of  the  system  engineering  effort  performed  during 
the  Conceptual  and  Definition  Phases.  The  earlier  detail  design 
efforts  which  were  primarily  directed  toward  apportionment  of 
reliability  requirements  to  CEI's  will  now  be  directed  toward 
development  of  an  acceptable  design  approach.  Reliability  factors 
of  the  preliminary  detail  designs  for  CEI's  will  be  developed  based 
on  the  reliability  requirements  of  the  approved  Part  1  Detail  Speci¬ 
fications. 

As  the  design  progresses,  new  requirements  will  be  identified, 
many  of  which  will  involve  areas  of  development  that  are  directly 
influenced  by  the  results  of  the  reliability  engineering  and  trade¬ 
off  activities.  For  example,  reliability  data  in  the  form  of  ex¬ 
pected  failure  rates  is  a  major  factor  in  developing  end  item 
maintenance  features,  as  well  as  in  allocating  maintenance  spares. 
In  addition,  early  reliability  engineering  and  analysis  inputs  are 
essential  in  preparing  for  subsequent  Preliminary  Design  Reviews 
(PDR),  Critical  Design  Reviews  (CDR),  and  indirectly  the  First 
Article  Configuration  Inspection  (FACI). 

Reliability  Specialist  Support  of  PDR  (13).  The  contractor's  pre¬ 
liminary  detail  designs  are  reviewed  on  an  incremental  basis  as 
the  preliminary  design  of  each  CEI  is  completed.  This  Prelimi¬ 
nary  Design  Review  (PDR)  is  performed  to  determine  that  the 
design  approach  is  feasible  and  sound,  and  that  the  performance 
requirements  specified  in  Part  I  Detailed  Specification  con  be 
met.  A  successful  PDR  normally  is  a  prerequisite  to  continuing 
with  detail  design  efforts. 

The  review  should  include  an  assessment  of  updated  reliability, 
modeling,  allocation  and  prediction  data,  as  well  as  reliability 


design  features.  (See  Chapter  6  for  a  discussion  of  reliability 
design  review.)  In  order  to  properly  evaluate  reliability  engineer¬ 
ing  factors,  a  reliability  specialist  should  be  included  on,  or 
available  as  a  consultant  to  the  PDR  representation  of  both  the 
contractor  and  procuring  activity. 

e .  Validate  Reliability  Requirement?;  of  Part  I  Deta.'I  Specifica t ion s 

(14) .  Based  on  the  findings  and  recommendations  of  the  PDR,  the 
contractor  usually  will  update  and  validate  the  Part  I  Detail  Speci¬ 
fications  before  proceding  with  his  detail  design  activities.  The 
reliability  requirements  of  the  specifications  will  be  of  particular 
interest  because  any  specification  revision  in  response  to  the  PDR 
can  effect  CEI  reliability  even  though  the  revision  is  performed  to 
correct  deficiencies  in  other  areas.  Therefore,  each  revision  to 
the  Part*  I  Detail  Specifications  should  be  evaluated  with  respect 
to  its  impact  on  CEI  and  system  reliability. 

f.  Provide  Reliability  Engineering  Support  to  Detail  Design  Activities 

(15) .  Following  the  validation  of  the  Part  D  Detail  Specifications 
as  a  result  of  PDR  actions,  the  contractor  initiates  a  concerted 
detail  design  effort  which  will  result  in  test  and  production  hardware 
and  facilities  meeting  the  specified  requirements. 

The  primary  function  of  the  contractor's  reliability  program  during 
the  detail  design  effort  is  that  of  monitoring  and  evaluating  the 
design  as  it  progresses  to  assure  that  the  specified  level  of  relia¬ 
bility  is  being  achieved.  This  should  include  a  continuing  effort  'o 
update  the  reliability  model  and  documentation  to  reflect  details  of 
design  as  they  are  defined,  and  to  perform  periodic  reliability  p’e  - 
dictions  to  detect  design  problem  areas  at  the  earliest  possible 
time.  In  addition,  the  reliability  specialist  should  provide  support 
in  the  application  of  reliability  improvement  techniques  such  as 
those  discussed  in  Chapter  11, 

g.  Prepare  Category  I  Test  Procedures  and  Perform  Tests  (18,  1Q  and 
20).  As  the  detail  design  progresses,  the  category  I  test  plan  is 
expanded  and  category  I  test  procedures  are  prepared  to  include 
detail  system,  subsystem,  CEI  and  component  reliability  tests. 

(See  Chapter  10.)  The  reliability  test  plans  and  procedures  are 
then  implemented  as  part  of  the  category  I  testing  to  comply  with 
the  quality  assurance  provisions  of  the  Parti  Detailed  Specifica¬ 
tions. 

The  contractor's  category  I  test  procedures  should  include  relia¬ 
bility  design  analyses  and  demonstration  to  determine  compliance 
with  the  quantified  reliability  requirements  as  specified  in  the 
Part  I  Detail  Specifications  (see  Exhibit  II  of  AFSCM  375-1).  In 
a  typical  case,  however,  the  reliability  demonstrations  should  be 
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conducted  concurrently  with  other  operational  tests  in  order  to 
reduce  the  total  cost  and  time  requirements  for  completing  the 
category  I  test  program. 

h.  Evaluate  Reliability  Engineering  Data  for  CDR  (21).  The  results 
of  the  detail  design  effort  are  reviewed  to  determine  the  adequacy 
of  the  design  in  meeting  the  requirements  of  the  Part  I  Detail 
Specification.  This  Critical  Design  Review  (CDR)  considers  re¬ 
liability  engineering  and  analysis  data  as  a  part  of  the  overall 
engineering  and  design  documentation  rather  than  as  a  separate 
item.  Therefore,  the  CDR  does  not  include  a  "reliability  review" 
as  such.  However,  because  of  the  importance  placed  on  relia¬ 
bility  achievement  as  a  development  objective,  and  in  view  of  the 
specialized  nature  of  reliability  documentation,  it  is  usually 
desirable  to  include  reliability  engineering  representation  on,  or 
available  for  consultation  to  the  CDR  group. 

The  CDR  results  in  formal  evaluation  and  identification  of  specific 
engineering  documentation  being  prepared  to  govern  full-scale 
production. 

i.  Review  Reliability- Critical  Aspects  of  the  Part  II  Detail  Specifi¬ 
cations  (23).  The  results  of  the  development  and  review  actions 
provide  the  Product  Configuration  Baseline  which  is  documented 
in  Part  II,  "Product  Configuration  and  Acceptance  Test  Require¬ 
ments"  of  the  Detail  Specification  (see  Exhibit  II  of  AFSCM  375-1). 
This  document  does  not  include  specific  reliability  requirements. 
However,  its  preparation  should  be  subjected  to  the  review  and 
approval  of  the  reliability  specialist  to  assure  that  reliability 
factors  of  the  design  have  not  been  compromised. 

j.  Provide  Reliability  Design  Review  Support  to  FACI  (25).  The  First 
Article  Configuration  Inspection  (FACI)  is  a  critical  inspection  of 
the  first  article  to  be  produced  in  accordance  with  the  Part  II  Detail 
Specifications.  This  formal  review  is  primarily  concerned  with 
production  design  characteristics  and,  as  such,  does  not  directly 
consider  the  reliability  aspects  of  the  design.  However,  minor 
discrepancies  between  the  article  as  produced,  and  the  specifica¬ 
tion  requirements  are  sometimes  resolved  by  means  ol  waivers  in 
specification  requirements.  Approval  of  any  such  waivers  should 
be  subject  to  the  review  and  approval  of  reliability  specialists  to 
assure  that  the  CEI  reliability  is  not  compromised. 

k.  Prepare  for  and  Conduct  Category  II  Reliability  Tests  (28  and 
35).  The  category  II  tests  are  intended  to  demonstrate  compli¬ 
ance  with  the  requirements  of  the  System  Specification  and  Part  I 
Detail  Specification,  both  of  which  contain  explicit  quantitative 

3-24 


reliability  requirements.  Therefore,  a  significant  portion  of 
category  II  testing  should  be  devoted  to  the  demonstration  of 
achieved  CEI,  subsystem  and  system  reliability,  Since  this  is 
the  only  demonstration  of  total  system  reliability  before  turn¬ 
over  of  the  system  to  the  using  agency,  the  reliability  demon¬ 
stration  conducted  during  category  II  testing  must  be  considered 
as  one  of  the  major  milestones  of  the  reliability  program.  It 
is  significant  that  the  category  II  Final  Test  Report  includes  a 
report  on  the  "functional  reliability  of  the  system,  "  and  that 
these  reliability  factors  are  given  as  much  weight,  in  the  report 
as  any  of  the  system  performance  test  results.  See  Chapter  5 
and  Data  Item  Number  T-1Z0  of  AFSCM/AFLC  M  310-1. 

l.  Support  Development  of  Category  III  Reliability  Tests  (38).  Cate¬ 
gory  III  testing  is  performed  by  the  using  command  during  the 
Operational  Phase,  to  assess  system  effectiveness  and  reliability 
in  the  intended  operational  environment.  Although  actual  testing 
begins  in  the  Operational  Phase,  planning  for  the  category  III 
tests  must  be  initiated  earlier  to  allow  for  proper  test  support 
and  scheduling.  The  responsibility  for  formulating  the  category 
III  test  plan  and  procedures  rests  with  the  using  command.  How¬ 
ever,  essential  support  in  all  technical  a  i  engineering  areas  is 
provided  by  the  SPO. 

Category  III  testing  normally  includes  system  effectiveness ,  oper¬ 
ational  readiness  or  other  form  of  demonstration  which  requires 
an  assessment  of  system  and  subsystem  reliability.  Therefore, 
reliability  activities  of  the  SPO  should  include  support  in  develop¬ 
ing  the  reliability  evaluation  and  demonstration  aspects  of  the 
category  III  test  plans  and  procedures. 

m.  Perform  Reliability  Analysis  During  Technical  Approval  Demon¬ 
stration  (42).  One  of  the  major  activities  of  the  contractor  in 
concluding  the  Acquisition  Phase  is  the  Technical  Approval  Demon¬ 
stration  (TAD),  which  is  a  demonstration  to  show  that  each  CEI, 
each  subsystem,  and  the  complete  system  are  acceptable  in  the 
configuration  intended  for  turnover.  Acceptability  of  the  sy.  m 
reliability  is  normally  measured  in  terms  of  a  calculated  quantity 
such  as  MTBF,  or  a  predicted  quantity  such  as  the  probability 

of  mission  success  at  a  random  point:  in  time.  In  any  case,  relia¬ 
bility  demonstration  and  prediction  data  as  developed  during  earlier 
testing  and  analysis  will  provide  a  major  input  to  the  TAD. 
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7. 


RELIABILITY  PROGRAM  MANAGEMENT  DURING  THE  OPERATIONAL 
PHASE 


Reliabililty  piograni  management  activities  performed  during  the  Oper¬ 
ational  Phase  should  be  directed  toward  assuring  the  turnover  of  an 
adequately  reliable  system  to  the  using  command  or  organization.  In 
general,  these  activities  will  include  the  completion  of  residual  relia¬ 
bility  engineering  and  testing,  supporting  the  category  III  test  program, 
performing  reliability  data  analysis  and  evaluation,  and  providing  relia¬ 
bility  engineering  support  for  modification  programs. 

The  responsibility  for  reliability  program  management  during  the 
Operational  Phase  is  transferred  from  the  System  Program  Director 
of  the  SPO  to  the  System  Support  Manager  of  AFLC.  This  transfer  of 
responsibility  is  not  abrupt,  but  rather  is  accomplished  on  an  incre¬ 
mental  basis  as  operating  units  of  the  system  are  accepted  by  the  user. 

Typically,  the  Acquisition  Phase  and  Operational  Phase  overlap  to  a 
considerable  degree.  The  acceptance  of  the  first  operating  unit  by  the 
using  command  initiates  the  Operational  Phase,  but  the  Acquisition 
Phase  i.-  not  concluded  until  the  last  operating  unit  is  delivered,  tested, 
and  turned  over. 

A  sequential  diagram  indicating  reliability  program  activities  of  the 
Operational  Phase  is  shown  in  Figure  3-4.  Activities  shown  in  this 
diagram  are  keyed  to  corresponding  blocks  of  Figure  9  of  AFSCM  375-4. 

The  reliability  program  activities  indicated  in  Figure  3-4  are  described 
in  the  following  discussion  of  Operational  Phase  activities.  The  numbers 
in  parentheses  following  each  subject  heading  also  refer  to  specific 
blocks  in  Figure  9  of  AFSCM  375-4. 

The  reliability  program  activities  summarized  below  are  directed 
toward  a  system  program  and  all  of  these  may  not  be  directly  applica- 
able  to  non-system  programs  in  all  cases.  However,  many  of  these 
activities  are  applicable  during  the  activation  and  operation  of  any 
new  equipment  item  even  though  it  is  not  identified  as  a  system  element. 
For  example,  reliability  engineering  support  (see  paragraph  7e)  should 
be  provided  for  modification  programs  associated  with  any  equipment 
in  the  operational  inventory.  In  addition,  failure  data  analysis  pro¬ 
grams,  as  mentioned  in  paragraph  7g,  are  important,  not  only  in 
evaluating  the  operation  of  a  new  equipment,  but  also  in  obtaining  data 
applicable  to  development  of  other  equipment  items. 

a.  Reliability  Engineering  Transierred  to  AFLC  (2).  Following  the 
contractor's  successful  completion  of  the  Technical  Approval 
Demonstration  (TAD),  and  using  command  acceptance  of  hardware 
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items  and  facilities  the  responsibility  for  operational  engineer¬ 
ing  for  the  respective  units  is  transferred  to  AFL.C. 


Operational  engineering  is  performed  to  resolve  service- revealed 
deficiencies,  and  to  investigate  other  operational  aspects  of  the 
performance  and  reliability  of  the  system.  Therefore,  an  impor¬ 
tant  activity  of  operational  engineering  should  be  the  analysis  of 
failure  data  obtained  via  the  Air  Force  Maintenance  Data  Collection 
System  (see  Chapter  7)  .  In  particular,  system  reliability  evalu- 
tions  should  be  performed  using  data  derived  from  analyses  of 
high  system  failures,  and  component/item  data  related  to  unsatis¬ 
factory  reliability  experience.  Due. to  the  emphasis  on  the  investi¬ 
gation  of  system  reliability  problems  it  is  apparent  that  reliability 
engineering  should  be  a  significant  portion  of  the  operational  engi¬ 
neering  function.  Therefore,  AFLC's  responsibility  for  operational 
engineering  includes  the  assumption  of  a  portion  of  the  system 
reliability  program  management  responsibility. 


Conclude  Category  II  Reliability  Testing  (4).  In  the  ideal  case 
category  II  testing  is  completed  prior  to  turnover  of  tae  first  unit. 
However,  in  many  practical  cases  category  II  testing  extends  into 
the  early  part  of  the  Operational  Phase  to  accommodate  tests  re¬ 
quiring  long  periods  of  data  gathering.  Reliability  demonstration 
is  one  of  the  most  common  of  such  long -duration  tests.  Quite  often, 
the  desired  level  <A‘  confidence  that  the  system  reliability  require¬ 
ment  is  met  can  only  be  established  by  accumulating  extensive 
operating  time. 


Update  Reliability  Fa ctors  in  Product  Configuratio n  Baseline  ( 6 ) . 
System  deficiencies  revealed  during  category  II  testing  are  cor¬ 
rected  by  updating  changes,  some  of  which  can  be  initiated  after 
the  start  of  the  Operational  Phase.  Because  of  the  extended  time 
required  for  reliability  demonstrations  during  category  II  testing, 
it  is  conceivable  that  updating  changes  extending  into  the  Operational 
Phase  will  involve  a  significant  amount  of  reliability  engineering. 
Therefore,  reliability  support  in  developing  changes  and  updating 
the  Product  Configuration  Baseline  should  continue  unitl  the  final 
change  has  been  completed. 


Provide  Reliability  Support  to  Category  III  Testing  (7).  Category 
III  testing  is  started  by  the  using  command  after  the  first  operating 
unit  has  been  accepted  and  after  all  operationally  critical  updating 
changes  have  been  made.  In  general,  category  III  tests  include 
some  form  of  reliability  or  system  effectiveness  demonstration 
that  will  be  performed  as  a  part  of  other  operational  testing  and 
evaluation.  Therefore,  a  separate  reliability  test  may  not  be 
required.  However,  due  to  the  specialized  methods  for  analyzing 
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reliability  data,  the  category  III  test  force  should  either  id e 

a  system  reliability  evaluation  specialist,  or  appropriate  spi 
ciaiized  support  should  be  provided  by  the  SPO. 

Provide  Reliability  Engineering  Suppor-  to  Modification  Pr  g_ram 
(8  and  12).  Deficiencies  revealed  during  catego-y  III  testir  g  are 
corrected  by  means  of  modifications  which  are  the  response  bility 
of  either  AFSC  or  AFLC  depending  on  the  extent  of  the  mudifi  a- 
tion.  AFR57-4  defines  two  general  classes  of  modifications  that 
are  of  concern  to  the  reliability  program.  These  are  class  IV 
modifications  to  correct  serious  deficiencies  and  class  V  modi¬ 
fications  to  provide  new  mission  capabilities. 

Class  IV  modifications  ti;at  are  within  the  capabilities  of  AFLC 
engineering  will  be  the  responsibility  of  the  AFLC  System  Support 
Manager  (SSM).  Class  IV  modifications  that  are  beyond  the  capa¬ 
bilities  of  AFLC  engineering,  and  all  class  V  modifications  arc 
assigned  to  AFSC.  In  the  latter  cases,  the  modification  develop¬ 
ment  is  managed  in  accordance  with  the  System  Program  Manage¬ 
ment  procedures  of  AFSCM  375-5,  beginning  at  the  appropriate 
point  in  the  Definition  or  Acquisition  Phase.  Such  modification 
programs  will  require  reliability  program  management  activities 
similar  to  those  discussed  previously  for  a  new  system  develop¬ 
ment  program. 

Perform  Reliability  Assessment  of  Change s  and  Modifications  (13, 
15  and  17),  Follow-up  development  tests  to  evaluate  updating 
changes  and  modifications  are  similar  to  category  I  or  II  tests, 
but  are  usually  on  a  reduced  scale.  They  are  performed  against 
specific  revisions  of  the  Design  Requirements  Baseline,  and 
emphasize  design  aspects  directly  related  to  the  change  or  modi¬ 
fication.  However,  other  tests  including  reliability  assessment 
are  required  to  assure  that  the  modification  did  not  adversely 
impact  on  other  system  characteristics.  Quite  often,  empirical 
confirmation  of  reliability  characteristics  will  not  be  practical 
during  follow-up  development  testing  due  to  the  time  required  to 
obtain  a  statistically  valid  reliability  measure.  In  this  case, 
analytical  procedures  should  be  employed  to  assess  the  effect  of 
the  modification  on  the  achieved  system  reliability. 

Continue  Failure  Data  Analysis  (11).  Data  are  collected  on  all 
operating  units  to  reflect  failure  experience  during  installation 
and  checkout  as  well  as  during  operation  by  the  using  command. 
Failure  occurring  during  checkout  are  reported  on  prescribed 
forms  such  as  AFSC  Form  258.  After  the  units  have  been  turned 
over  to  the  using  command  the  Maintenance  Data  Collection  System 
as  defined  in  AFM66-1  is  instituted  and  failure  data  are  reported 
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using  the  AFTO-200  series  forms  as  described  in  Chapter  7. 

Failure  data  feedback  and  analysis  is  a  continuing  activity  that 
may  not  terminate  until  disposition  of  the  system.  These  data 
are  a  primary  input  for  system  reliability  analyses  and  other 
studies  that  are  performed  throughout  the  operational  life  of  the 
system,  and  that  provide  historical  reliability  data  essential  to 
the  development  of  future  systems.  The  analysis1  and  evaluation 
of  failur  data  is  one  of  the  most  significant  activities  of  the  relia  - 
abiiitv  program  following  transition  of  the  systen  ngineering 
responsibility  from  AFSC  to  AFLC. 

h.  T ransition  P  '  lability  Program  Management  to  AFLC  (14  through 
28).  At  the  conclusion  of  the  final  Acquisition  Phase  activities, 
all  system  management  functions  will  have  been  transitioned  from 
AFSC  to  AFL.C.  By  this  time,  total  Air  Force  engineering  respon¬ 
sibility  will  liave  been  assumed  by  AFLC.  Therefore,  any  long¬ 
term  reliability  engineering  function,  such  as  failure  data  analysis 
and  operational  reliability  evaluation  will  become  the  responsibility 
of  ..FLC  engineering.  Thus,  AFLC  will  assume  the  continuing 
reliability  program  management  responsibility  for  the  system. 

i.  Continue  to  Support  Operational  Reliability  Testing  (2  9).  Category 
III  testing  is  completed  by  the  using  command  in  accordance  with 
the  previously  prepared  category  III  test  plan.  However,  this  does 
not  conclude  the  system  testing  activity.  A  program  to  perform 
operational  testing  is  continued  until  the  system  has  been  exercised 
under  various  conditions  and  loads,  and  within  the  constraints  of 

a  variety  of  missions.  The  system  is  tested  on  an  incremental 
basis  until  all  system  elements  have  demonstrated  acceptable  per¬ 
formance  in  a  variety  of  operating  environments. 

Such  continued  operational  testing  is  justified  by  providing  means  for: 

,  Training  operational  personnel  and  evaluating  their  perform¬ 
ance.  (Note:  Operating  personnel  are  a  part  of  the  total  system 
configuration,  and  are  considered  as  basic  elements  in  evalu¬ 
ating  operational  reliability.  ) 

.  Assessing  system  capability  in  view  of  changing  threats. 

.  Identifying  the  need  for  system  modification  or  a  new  system. 
(This  is  one  of  the  inputs  to  the  pre-con-.eputal  requirements 
and  planning  studies  for  future  generations  of  systems.  ) 

.  Permitting  evaluation  of  the  impact  of  new  interfacing  systems 
or  changes  to  existing  interfacing  systems. 

.  Providing  measures  of  system  performance  and  reliability 
under  operational  stress. 
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The  need  for  a  continuing  reliability  program  in  realizing  this 
last  advantage  is  obvious.  However,  ether  areas,  such  as 
assessing  system  capability,  identifying  the  need  for  modifica¬ 
tion,  and  evaluating  interfacing  impact  also  involve  the  con¬ 
sideration  and  evaluation  of  r fillabilit'1  factors.  It  is  apparent, 
therefore,  that  reliability  program  activities  will  continue 
throughout  the  Operational  Phase  and  into  the  Conceptua,  Phase 
of  'he  next  generation  of  ystems,  thereby  completing  the  cycle. 


\ 
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CHAPTER  4 


RELIABILITY  ENGINEERING  MANAGEMENT 


1.  INTRODUCTION 

Chapter  3  has  presented  an  overall  discussion  of  the  objectives  and 
activities  of  Reliability  Program  Management  in  relation  to  the  en¬ 
tire  spectrum  of  system  program  management,  including  significant 
activities  in  the  areas  of  configuration  control,  systems  ehgineering, 
and  test  and  deployment  which  directly  relate  to  hardware  develop¬ 
ment. 

The  ultimate  objective  of  the  reliability  program  is  the  acquisition 
of  an  appropriately  reliable  system.  The  required  level  of  reliability 
is  stated  in  the  system  specification  as  a  part  of  the  configuration  con¬ 
trol  activity,  and  the  achieved  reliability  is  verified  as  a  major  activity 
of  the  test  program.  However,  configuration  control  and  testing  cannot 
"achieve"  reliability.  Reliability  achievement  ;s  the  unique  responsi¬ 
bility  of  reliability  engineering.  In  fact,  without  an  effective  reliability 
engineering  program,  the  reliability  assurance  activities  associated 
with  the  configuration  control  and  testing  program  will  be  meaningless. 
Therefore,  an  effective  reliability  engineering  program  is  an  essential 
aspect  of  the  total  systems  engineering  process.  Furthermore,  an 
effective  reliability  engineering  program  can  only  be  assured  by  the 
timely  and  appropriate  management  activities  throughout  the  life  cycle 
of  system  development. 

Specific  considerations  in  the  management  of  an  effective  reliability 
engineering  program  are  discussed  in  this  chapter.  This  discussion 
is  presented  within  the  concepts  of  the  Systems  Engineering  Manage¬ 
ment  Procedures  of  AFSCM375-5,  but  also  relates  specific  manage¬ 
ment  activities  to  other  less  comprehensive  development  programs 
where  the  complete  systems  engineering  approach  is  not  warranted. 

Reliability  engineering  encompasses  a  variety  of  engineering  design 
and  analysis  disciplines  which  are  applicable  at  all  levels  of  system 
design  and  development  as  an  integral  part  of  the  total  system  engineer¬ 
ing  process.  As  a  defined  program,  reliability  engineering  begins 
with  the  initiation  of  the  system  engineering  activities  performed  by 
the  SPO  cadre  during  the  conceptual  transition  stage  of  the  Ccnceptual 
Phase.  This  does  not  imply  that  important  reliability  program  activi¬ 
ties  are  not  performed  earlier.  In  fact  reliability  achievement  can 
be  a  major  objective  of  the  technological  development  activities  of  the 
early  Conceptual  Phase.  However,  these  earlier  activities  are  generally 
developmental  rather  than  engineering-oriented  and,  therefore,  will 
not  be  discussed  in  this  chapter. 
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Reliability  engineering  typically  is  initiated  as  an  element  of  the  sys¬ 
tem-engineering  activities  during  Conceptual  Transition,  and  con¬ 
tinues  throughout  the  Definition  ar.d  Acquisition  phases  and  into  the 
Operational  phase.  In  general,  reliability  engineering  encompasses 
two  areas  of  activity:  (1)  reliability  achievement  activities  which 
include  part  selection,  derating,  redundancy  design,  and  other  en¬ 
gineering  activities  directed  toward  the  design  of  a  reliable  product; 
and  (2)  reliability  assurance  activities  which  include  the  allocation, 
specification  and  verification  of  compliance  with  reliability  require¬ 
ments. 

It  is  the  responsibility  of  reliability  engineering  management  to  assure 
an  effective  reliability  engineering  program  during  all  phases  of  the 
system  life  cycle.  In  view  of  this,  the  balance  of  this  chapter  is  directed 
toward  a  summary  of  reliability  engineering  activities  that  are  per¬ 
formed  during  the  various  phases  of  a  typical  system  development  pro¬ 
gram.  This  discussion  is  presented  within  the  concepts  of  the  Systems 
Engineering  Management  Procedures  of  AFSCM  375-5.  However,  many 
of  the  activities  described  are  applicable  to  any  development  program, 
including  hardware  procurement  and  experimental  model  development 
programs  such  as  those  performed  in-house  by  RADC,  and  which  do 
not  fall  within  the  system  engineering  concepts, 

2.  RELIABILITY  ENGINEERING  MANAGEMENT  DURING  CONCEPTUAL 
TRANSITION 

The  responsibility  for  system  reliability  engineering  management 
during  the  Conceptual  Phase  is  normally  assumed  by  the  System  Pro¬ 
gram  Manager  upon  receipt  of  the  SOR/OSR/ADO  and  initiation  of 
Conceptual  Transition.  Thus,  the  SPO  cadre  that  is  established  at 
this  time  should  include,  as  a  part  of  the  systems  engineering  activity, 
an  element  (at  least  one  person)  having  the  overall  responsibility  for 
initiating  and  conducting  the  reliability  engineering  program.  This 
element  will  assume  the  reliability  engineering  responsibilities  under 
the  Deputy  Director  for  Engineering  of  the  SPO  upon  initiation  of  the 
Definition  Phase. 

The  primary  objective  of  the  reliability  engineering  program  activities 
during  Conceptual  Transition  is  to  establish  the  overall  system  reli¬ 
ability  requirements,  allocate  these  requirements  to  the  major  system 
functions  and  from  this  develop  the  reliability  requirements  input  for 
the  PTDP.  Several  specific  activities  have  been  defined  for  meeting 
this  objective.  These  activities  are  summarized  in  Table  IV- 1,  where 
specific  reliability  engineering  activities  are  identified  together  with 
the  purpose  of  the  activity,  general  source  of  input  data,  type  of  out¬ 
put,  and  specific  use  of  output  data.  The  engineering  techniques  used 
and  other  information  relating  to  each  activity  identified  in  Table  IV- 1 
are  summarized  in  the  following  paragraphs. 
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Table  IV-I.  Aii  Force  SPO  Cadre  Reliability  Engineering  A  ctivities- Conceptual  Phase 


2.  1  Review  nf  Source  Documentation. 


The  initial  activity  of  the  Conceptual  Transition  phase  is  that  of 
reviewing  available  source  documentation  to  identify  all  infor¬ 
mation  and  data  that  wdl  relate  to  the  subsequent  eliability  en¬ 
gineering  activities.  As  a  minimum,  this  revie'''  should  be  directed 
toward  identifying  sources  of  data  relating  to  the  following  factors: 

a.  Mission  profile  descriptions,  indicating  factors  from  which 
system  reliability  requirements  are  developed.  This  includes 
data  relating  to  system  operating  time  requirements  in  each 
mode  of  operation,  and  any  other,  factors  that  will  aid  in  de¬ 
fining  the  time  constraints  that  will  be  imposed  on  the  system. 

b.  System  functional  requirements  for  each  defined  mission.  This 
includes  information  identifying  system  functional  configuration 
in  each  mode  of  operation,  and  an  assessment  of  the  probable 
operational  consequences  in  the  event  of  loss  of  any  given  sys¬ 
tem  function. 

c.  Any  constraints  that  may  limit,  or  place  excessive  demands  on 
system  reliability.  This  includes  such  constraints  as  maintain¬ 
ability  limitations,  and  other  factors  that  may  impact  on  the 
definition  and  allocation  of  reliability  requirements. 

d.  System  effectiveness,  availability,  operational  reliability,  and 
other  requirements  of  the  system  that  include  reliability  as  a 
parameter,  and  which  will  facilitate  quantification  of  system 
reliability  requirements. 

2.  2  Quantification  of  System  Reliability  Requirements. 

Following  the  effort  to  gather  and  review  source  data,  the  reliability 
engineering  program  begins  with  the  initial  quantification  of  the  gross 
reliability  requirements  for  the  system.  At  times,  system  reli¬ 
ability  requirements  are  stated  in  the  SOR/OSR/ADO  documents  and 
need  only  to  be  updated  and  verified  at  this  time.  In  general,  however, 
the  system  reliability  requirements  must  be  derived  by  interpretation 
of  qualitative  statements  concerning  the  intended  mission,  and  gross 
quantifications  of  system  effectiveness,  operational  reliability,  or 
availability  requirements.  These  latter  factors  are  functions  of  re¬ 
liability,  maintainability,  and  other  parameters  which  must  be  con¬ 
sidered  in  establishing  the  design  goals  for  the  system.  However, 
there  is  considerable  latitude  for  trade-off  between  the  various 
parameters  before  the  system  design  goals  can  be  optimized. 

The  task  of  quantifying  the  overall  system  reliability  requirements 
generally  involves  a  trade-off  between  maintainability  (mean  time 
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to  restore,  maximum  allowable  repair  time,  probability  of 
restoration  within  a  given  time,  etc.)  and  reliability  (mean 
time  between  failure,  probability  of  survival,  etc.  ),  together 
with  other  parameters,  including  operational  characteristics 
and  cost  factors.  Thus,  this  initial  development  of  systen 
design  goals  involves  a  gross  allocation  of  the  various  param¬ 
eters  contributing  to  the  achievement  of  system  effectiveness. 

Techniques  employed  in  the  trade-off  between  reliability, 
maintainability  and  other  parameters  in  establishing  reliability 
specification  requirements  are  reviewed  in  Chapter  6.  Some 
additional  techniques  for  interpreting  mission  requirements  in 
terms  of  reliability  goals  are  summarized  in  Chapter  8. 

2.  3  Development  of  Reliability  Block  Diagram  and  Model. 

The  initial  reliability  block  diagram  and  mathematical  model,  as 
developed  during  Conceptual  Transition,  forms  the  basis  for  all 
subsequent  reliability  engineering  program  activities.  Therefore, 
this  task  of  developing  the  initial  block  diagram  and  model  is  one 
of  the  more  important  of  the  early  reliability  engineering  activities. 

The  initial  reliability  block  diagram  is  generally  structured  to 
define  functional  relationships,  and  is  essentially  a  refinement 
of  the  top-level  and  first- level  functional  diagrams  which  are 
modified  to  include  the  gross  reliability  requirement,  and  the 
results  of  some  early  reliability  predictions  that  can  be  performed 
at  this  time.  As  the  reliability  engineering  program  progresses, 
the  model  is  revised  and  updated  until  reliability  factors  relating 
to  individual  elements  of  system  hardware  can  be  related  to  overall 
system  reliability,  as  well  as  to  the  other  parameters  of  system 
effectivenes  s . 

2.4  Initial  Reliability  Allocations. 

The  gross  reliability  allocations  performed  during  the  conceptual 
phase  are  directed  toward  the  assignment  of  a  feasible  reliability 
goal  for  each  function  as  defined  on  the  functional  diagram.  These 
allocations  are  performed  using  preliminary  reliability  prediction 
techniques,  such  as  the  reliability  prediction  by  function  techniques 
discussed  in  Chapter  9,  and  are  verified  using  the  initial  reli¬ 
ability  model  to  assure  that  the  levels  of  reliability,  as  allocated, 
are  appropriate  in  relation  to  the  required  total  system  reliability. 

The  results  of  the  reliability  allocation  are  used  in  developing  re¬ 
liability  design  requirement  statements  to  be  included  in  the  initial 
requirements  allocation  sheets  (RAS).  These  documents  are  prepared 
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as  a  part  of  the  system  engineering  activity  and  provide  one  of 
the  primary  technical  inputs  to  the  PTDP.  The  results  of  the 
allocation  studies  are  also  important  inputs  to  the  reports 
describing  Conceptual  Phase  trade  study  activities. 

2.5  Reliability  Design  Requirements. 

The  final  major  activity  of  the  reliability  engineering  program 
during  Conceptual  Transition  includes  preparing  the  reliability 
design  requirements  that  will  ultimately  appear  in  the  System 
Specifications.  At  this  stage,  these  requirements  are  stated  in 
specification  language  and  are  incorporated  into  Design  Sheets 
which  become  a  part  of  the  PTDP.  Thus,  the  reliability  design 
requirements  form  a  part  of  the  Program  Requirements  Baseline 
governing  the  activities  of  the  Definition  Phase. 

3.  RELIABILITY  ENGINEERING  MANAGEMENT  DURING  PHASE  A  OF 
THE  DEFINITION  PHASE 

Upon  initiation  of  the  Definition  Phase,  the  full  SPO  is  established  and 
the  Deputy  Director  for  Engineering  assumes  responsibility  for  all  sub¬ 
sequent  system  engineering  activities.  This  includes  all  activities 
associated  with  reliability  engineering. 

Phase  A  of  the  Definition  Phase  includes  those  system  engineering 
activities  by  the  SPO  that  are  necessary  in  preparing  a  request  for 
proposals  (RFP)  for  Contract  Definition  (Phase  B).  Additional  system 
engineering  is  performed  by  each  of  the  competing  contractors  in  pre¬ 
paring  their  proposals,  and  final  engineering  analyses  are  performed 
by  the  SPO  in  evaluating  the  various  proposals  in  preparation  for  award 
of  the  Definition  Contract. 

The  reliability  engineering  activities  as  discussed  here  are  those  per¬ 
formed  by  the  SPO  in  support  of  the  RFP  development,  and  in  evaluating 
proposals  that  are  prepared  in  response  to  the  RFP.  These  activities 
are  summarized  in  Table  IV- 2.  The  engineering  techniques  used,  and 
other  information  relating  to  each  activity  are  summarized  below.  In 
addition,  typical  reliability  engineering  activities  that  might  be  per¬ 
formed  by  a  prospective  definition  contractor  in  preparing  his  proposal 
are  reviewed  with  reference  to  the  impact  on  the  SPO  reliability  en¬ 
gineering  activities  in  concluding  Phase  A. 

3.  1  Expanding  Reliability  Requirements. 

Following  the  authorization  to  begin  the  Definition  Phase,  a  signifi¬ 
cant  amount  of  system  engineering  is  necessary  in  determining 
additional  design  requirements  and  performing  trade-off  studies  as 
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Table  IV -2.  Air  Force  SPO  Reliability  Engineering  A ctivities - Phas e  A  Of  Definition  Phase 


required  for  the  preparation  of  the  initial  System  Performance/ 

Design  Requirements  General  Specification  (System  Specification). 

These  activities  result  in  expanding  the  functional  diagrams  to  in¬ 
clude  second-level  functions,  and  respective  expansion  of  the  RAS 
and  Design  sheets.  As  these  systems  engineering  activities  pro¬ 
gress,  the  reliability  requirements  must  also  be  updated  and  ex¬ 
panded  as  appropriate. 

The  reliability  engineering  efforts  required  to  support  these  sys¬ 
tem  engineering  activities  includes  the  expansion  of  the  reliability 
block  diagram  and  model  to  reflect  the  additional  information  pre¬ 
sented  in  the  updated  functional  diagrams.  The  expanded  model  is 
then  utilized  for  further  allocating  the  quantified  reliability  design 
requirements  to  the  newly  defined  elements  of  the  system.  In  gen¬ 
eral,  this  allocation  is  still  related  to  functional  rather  than  physical 
subdivisions  of  the  system  and.  thus,  will  involve  the  application  of 
reliability  prediction  procedures  similar  to  those  used  in  the  initial 
allocation.  In  some  cases,  however,  system  functions  may  be  de¬ 
fined  to  the  extent  that  prediction  procedures  considering  more  de¬ 
tailed  parameters  such  as  equipment  complexity  can  be  utilized. 

The  results  of  this  refinement  and  expansion  of  the  reliability  alloca¬ 
tion  are  used  in  providing  reliability  design  requirements  for  the 
expanded  RAS's  and  Design  sheets. 

3.2  Determining  Additional  Reliability  Engineering  Requirements. 

The  reliability  engineering  activities  during  the  early  stages  of 
Phase  A  not  only  provide  updated  system  reliability  requirements 
information,  but  also  permit  the  identification  of  the  additional  re¬ 
liability  engineering  activities  that  will  be  necessary  before  the 
system  is  completely  defined.  In  general,  these  additional  require¬ 
ments  can  be  categorized  as  either  (1)  those  that  can  be  expected 
to  be  satisfied  as  a  result  of  the  proposal  development  efforts  of 
the  contractors  who  are  competing  for  the  Definition  contract,  and 
(2)  those  that  will  require  the  more  extensive  engineering  efforts 
of  the  actual  contractor  Definition  (Phase  B)  activities.  These 
additional  requirements  will  be  included  in  the  proposal  preparation 
instructions  and  statement  of  work  accompanying  the  Phase  B  RFP. 

3.  3  Preparing  Reliability  Requirements  Input  for  Initial  System  Specifications. 

The  updated  system  reliability  requirements  are  prepared  in  the 
appropriate  specification  language.  This  will  involve  the  application 
of  techniques  such  as  those  described  in  Chapter  6  and  10  for 
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specifying  reliability,  and  will  be  prepared  in  the  format  required 
for  direct  use  in  the  System  Specification.  (See  Exhibit  I  of 
AFSCM  375-  1.  ) 

3.4  Initial  Development  of  Reliability  Test  Plan. 

A  reliability  test  plan,  compatible  with  the  system  reliability  re¬ 
quirements  is  initiated  during  this  phase,  and  is  refined  as  the 
system  reliability  requirements  are  updated.  This  initial  plan 
will  identify  the  general  testing  requirements,  as  applicable  dur¬ 
ing  the  Category  f  and  Category  II  testing  programs,  and  to  the 
extent  required  for  guiding  the  subsequent  test  plan  development 
efforts  of  the  Definition  Phase  contractors.  The  initial  reliability 
test  plan  should  be  considered  in  preparing  the  wording  for  the 
reliability  test  requirements  paragraph  of  the  System  Specification. 
(See  Exhibit  I  of  AFSCM  375-1.  )  Engineering  considerations  in 
the  development  of  reliability  test  plans  are  discussed  in  detail  in 
Chapter  10. 

3.  5  Reliability  Inputs  for  Phase  B  RFP. 

The  results  of  the  SPO  reliability  engineering  activities  during 
Phase  A  provide  the  information  necessary  for  preparing  the  re¬ 
liability  requirements  input  to  the  Phase  B  RFP.  As  a  minimum, 
this  should  include  the  preparation  of  definitive  statements  con¬ 
cerning: 

a.  Reliability  requirements,  constraints,  and  other  considerations 
in  conducting  trade  studies  during  Contractor  Definition. 

b.  Test  program  requirements  for  demonstrating  reliability 
factors  as  allocated  to  the  defined  system  elements. 

c.  Any  incentive  recommendations  that  would  be  of  concern  to 
the  contractors'  reliability  engineering  efforts.  This  includes 
the  identification  of  high-risk  areas  to  serve  as  a  basis  for 
development  of  incentive  provisions. 

3.6  Contractors'  Reliability  Engineering  Activities  During  Phase  A. 

During  Phase  A  of  the  Definition  Phase,  the  prospective  definition 
contractors  will  perform  a  series  of  system  engineering  studies 
and  analyses  as  necessary  to  provide  input  to  their  Contract  Defi¬ 
nition  proposals  which  are  prepared  in  response  to  the  RFP.  These 
engineering  activities  will  be  governed  by  the  specific  requirements 
of  the  RFP  and  System  Specification,  and  usually  will  include  a 
significant  reliability  engineering  effort.  Some  of  the  typical  activities 
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are  summarized  below.  These  actions  are  discussed  to  illustrate 
the  contractors'  reliability  engineering  iterations  rather  than  to 
identify  specific  procedures  that  the  contractor  is  required  to 
follow . 

a.  Review  RFP  Documentation.  The  initial  efforts  of  the  con¬ 
tractors'  reliability  engineers  should  be  to  review  the  RFP, 
and  all  documents  that  accompany  the  RFP  to  identify  all  sys¬ 
tem  reliability  design  requirements,  and  establish  an  approach 
to  the  development  of  the  reliability  engineering  and  test  pro¬ 
gram. 

b.  Expand  and  Update  System  Reliability  Data.  The  updated  re¬ 
liability  block  diagram,  requirement  allocation  sheets,  sys¬ 
tem  specification,  and  other  data  relating  to  the  reliability  en¬ 
gineering  effort  should  be  expanded,  refined,  and  updated  to 
the  extent  possible  with  relation  to  the  system  reliability  re- 
qui:.  ements . 

v. .  Verify  Reliability  Requirements.  Based  on  the  updated  data, 
the  contractor  should  verify  the  reliability  requirements,  in¬ 
cluding  test  program  plans  and,  where  necessary,  should  in¬ 
terpret  such  requirements  in  terms  of  his  proposed  approach. 
Any  apparent  or  real  deviation  from  the  requirements  of  the 
RFP  should  be  fully  explained  and  justified. 

d.  Provide  Inputs  to  Contractor's  Proposal.  The  reliability  en¬ 
gineering  activities  are  concluded  with  the  preparation  of  in¬ 
puts  to  the  contractor's  proposal  for  the  Phase  B  Definition 
effort.  Specific  inputs  should  include  at  least  the  following: 

A  discussion  to  demonstrate  the  contractors  full  under¬ 
standing  of  the  system  and  end  item  reliability  require¬ 
ments  and  reliability  program  objective. 

A  reliability  program  plan,  summarizing  the  contractor's 
approach  to  the  reliability  engineering  and  analysis  tasks 
during  the  Contractor  Definition  program. 

4.  RELIABILITY  ENGINEERING  MANAGEMENT  DURING  PHASE  B  OF 
THE  DEFINITION  PHASE 

Phase  B  of  the  Definition  Phase  involves  essentially  contractor  activity 
with  the  SPO  system  engineering  activity  providing  guidance  and  support 
to  the  participating  contractors,  and  periodically  reviewing  the  results 
of  the  system  engineering  activities.  Formal  reviews  of  system  require¬ 
ments  and  sy-tem  design  are  performed  periodically  during  a  typical 
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contractor  Definition  program.  Three  of  these  are  associated  with 
the  development  of  the  Part  I  Detail  Specifications  for  CEI's,  and 
directly  involve  the  participation  of  the  SPO  reliability  engineering 
activity.  Other  reviews  are  concerned  with  the  parallel  develop¬ 
ment  of  end  item  maintenance  design,  and  are  generally  of  secondary 
interest  to  the  reliability  engineering  program.  The  three  reviews 
requiring  direct  participation  of  the  reliability  engineering  activity  of 
the  SPO  are  summarized  in  Table  IV- 3.  Specific  activities  in  relation 
to  a  typical  definition  program  are  reviewed  in  the  following  paragraphs. 

4.  1  System  Requirements  Review  of  Operation's  Functions  Development. 

The  initial  review  by  the  SPO  is  performed  to  evaluate  the  contractors 
effort  to  define  the  system  requirements  in  terms  of  operations  func¬ 
tional  diagrams,  RAS's  and  time-line  sheets.  The  contractors'  re¬ 
liability  engineering  activities  to  this  point  will  typically  include  the 
development  of  refined  and  updated  RAS's,  and  design  sheets,  and 
the  performance  of  related  trade  studies.  These  activities  will  have 
been  centered  around  an  updated  and  refined  reliability  block  diagram 
and  mathematical  model,  which,  in  turn,  will  reflect  the  latest  ex¬ 
pansion  of  the  functional  diagram.  During  the  SPO  review  of  docu¬ 
mentation  generated  during  these  contractor  activities,  particular 
attention  should  be  given  to: 

a.  Verifying  the  allocations  of  reliability  and  the  means  by  which 
such  allocations  were  performed. 

b.  Assuring  the  completeness  and  accuracy  of  reliability  associated 
functions. 

c.  Verifying  reliability  block  diagrams  and  mathematical  models. 

d.  Assuring  the  adequacy  of  the  contractor's  design  and  hardware 
concepts . 

4.  2  System  Requirements  R  a  view  of  End  Item  Selection. 

The  next  se  'ies  of  contractor  reliability  engineering  activities  are 
directed  toward  the  development  of  detailed  reliability  require¬ 
ments  for  specific  end  items  as  the  various  items  are  selected. 

The  systems  engineering  activities  associated  with  this  selection 
include  a  series  of  trade  studies  and  the  development  of  design  re¬ 
quirements  for  each  CEI.  The  results  of  these  activities  are  docu¬ 
mented  by  means  of  expanded  and  updated  RAS's,  schematic  block 
diagrams,  and  design  sheets,  and  by  means  of.  trade  study  reports 
as  appropriate.  In  reviewing  these  documents,  the  SPO  reliability 
engineering  specialist  should  give  particular  attention  to: 
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Table  IV-3.  Air  Force  SPO  Reliability  Engineering  Activities- Phase  B  Of  Definition  Phase 
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Assuring  that  reliability  requirements  for  end  items  are 
stated  in  engineering  terms,  and  are  compatible  with  the 

me„V,r  7  devel°pment  System  function  reliability  require- 


b.  Assuring  that  each  design  sheet  includes  appropriate  reli- 
ability  design  and  test  requirements. 

c.  Assuring  that  the  reliability  test  and  demonstrations  are 
consolidated  w„h  the  other  Category  I  and  II  tests  to  'he 
maximum  extent  possible. 

4.  3  System  Design  Review. 

The  final  review  of  interest  to  the  SPO  reliability  engineering 
activity  is  concerned  with  the  results  of  the  reliabilitv  e  • 
activities  performed  during  the  development  of  the  Part  I  DetaiV^ 
Specifications  for  the  CEI's.  In  a  typical  system8  engineering 
program  the  contractor  will  have  performed  a  series  of  iterative 
engineering  activities  directed  toward  the  definition  of  design  re 

resuRmeif  end  items’  and  wili  have  documented  the 

results  of  these  activities  by  means  of  further  updated  and  ex 

panded  RAS  s,  and  schematic  block  diagrams,  in  addition  th* 

design  sheets  will  have  been  translated  into  the  initial  Pari  I 

AFSCM  375  lC)  S  C°ntraCt  6nd  it6mS  (SGe  ExMbit  11  of 

The  primary  purpose  of  the  reliability  engineering  input  to  this 
design  review  is  to  determine  whether  the  reliability  require 
ments  as  presented  in  the  Detail  Specifications  are  valid  and 
are  in  consonance  with  the  program  requirements  baseline  as 
o^ginaHy  estaohshed  by  the  System  Specification.  This  review 
should  include  at  least  the  following; 

a.  An  evaluation  to  assure  that  the  reliability  design  require 
ments,  as  specified  in  the  Part  I  Detail  Specifications  are' 
consistent  with  the  system  development  objectives,  and  are 
adequate  and  valid  as  a  design  requirement. 

b.  An  evaluation  of  the  reliability  test  and  analysis  require 
ments  as  stated  in  the  Part  I  Detail  Specifications.  This 
should  verify  that  a  reliability  test  and  analysis  require 
men  iG  specified  for  each  reliability  design  requirement 
and  that  the  test  plan  and  acceptance  criteria  will  verify  ’ 
compliance  within  the  desired  level  of  confidence.  (See 
Chapter  10.) 
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RELIABILITY  ENGINEERING  MANAGEMENT  DURING  PHASE  C 
OF  THE  DEFINITION  PHASE 


Phase  C  of  the  Definition  Phase  is  concerned  with  the  SPO's  review 
and  evaluation  of  the  definition  contractors'  final  reports  and  pro¬ 
posals  for  the  acquisition  phase  development  program.  This  review 
and  evaluation  is  directed  toward  determining  the  technical  soundness 
<if  the  defined  system,  assuring  the  adequacy  of  the  identification  of 
high  risk  areas,  and  determining  the  degree  to  which  the  technical 
tasks  specified  in  the  RFP  have  been  accomplished.  In  addition,  when 
major  advantage  would  accrue,  an  engineering  synthesis  is  made  of 
the  best  features  of  each  proposed  system  to  obtain  an  optimum  sys¬ 
tem  within  the  overall  performance,  cost,  and  schedule  requirements 
and  proprietary  limitations. 


The  reliability  engineering  activities  during  this  phase  include  a  final 
review  and  evaluation  of  the  reliability  engineering  aspects  of  each 
final  report  and  proposal  (see  Table  IV-4).  This  will  include  evalu¬ 
ation  of  the  reliability  requirements  and  test  provisions  as  stated  in 
the  following  documentation: 


a.  The  Updated  System  Specification. 


Updated  and  expanded  reliability  block  diagrams  and  mathematical 
models  (in  relation  to  updated  functional  block  diagrams). 


Requirements  allocation  sheets. 


Part  I  Detail  Specifications. 


Contractor's  reliability  program  management  plan. 


Contractor's  reliability  test  and  demonstration  plan. 


Contractor's  proposal  for  the  Acquisition  Phase,  including  reli¬ 
ability  engineering  techniques  and  procedures  on  his  statement 
of  work,  and  related  schedules,  costs,  incentive  features,  etc. 


The  results  of  this  evaluation,  synthesis,  and  supplementation  of  con¬ 
tractor's  results  are  reflected  in  the  revised  PTDP,  which  now  be¬ 
comes  the  Proposed  System  Packaging  Plan  (PSPP).  This  document 
provides  the  technical  input  to  the  Design  Requirements  Baseline 
governing  the  system  development  activities  of  the  Acquisition  Phase. 


4-  14 


Table  IV-4.  Air  Force  SPO  Reliability  Engineering  Activities-Pnase  C  Of  Definition  Phase 
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6.  RELIABILITY  ENGINEERING  MANAGEMENT  DURING  THE 
ACQUISITION  PHASE 

The  purpose  of  the  Acquisition  Phase  is  to  accomplish  the  delivery, 
installation  and  checkout  of  system  elements,  and  their  integration 
into  an  operable  system.  In  effect,  the  Acquisition  Phase  is  divided 
into  two  overlapping  efforts;  development  and  production.  The  de¬ 
velopment  effort  is  a  continuation  of  the  system  design  activity  of  the 
Definition  Phase,  and  continues  through  the  final  development  and 
approval  of  the  Part  II  Detail  Specifications  for  Contract  End  Items 
(CEI's).  This  document  establishes  the  technical  requirements  for 
the  Product  Configuration  Baseline  that  governs  production.  The 
production  effort  includes  the  actual  production  of  items  on  contract, 
together  with  the  installation,  checkout  and  acceptance  testing  of  all 
such  items. 

The  reliability  engineering  activities  during  the  Acquisition  Phase  are 
included  as  a  part  of  the  development  contractor's  system  engineering 
and  design  engineering  responsibilities.  Therefore,  the  SPO's  re¬ 
liability  engineering  activities  take  on  a  role  of  monitoring  and  review¬ 
ing  the  contractor's  reliability  design  review  and  reliability  test  and 
evaluation  efforts  to  assure  that  the  design  is  meeting  the  specification 
requirements . 

6.  1  Reliability  Engineering  Support  of  Design  Reviews. 

During  the  Acquisition  Phase,  the  contractor  is  required  to  per¬ 
form  a  number  of  formal  system  requirements  and  design  reviews 
directed  toward  verification  of  operations  and  maintenance  equip¬ 
ment  and  facilities  design.  The  particular  reviews  that  are  most 
significant  in  the  system-  development  engineering  effort,  and  that 
are  of  major  concern  to  the  reliability  engineering  activity  are 
the  Preliminary  Design  Reviews  (PDR),  the  Critical  Design  Re¬ 
views  (CDR),  and  the  First  Article  Configuration  Inspections 
(FACI).  In  general,  the  responsibility  of  the  SPO  reliability  en¬ 
gineering  activity  preceding  and  during  these  reviews  includes 
the  following: 

/ 

-  Reviewing  previously  developed  engineering  data  to  identify 
reliability  engineering  items  that  should  be  included  in  the 
review. 

-  Evaluating  contractor  prepared  review  schedules  ana  agenda  to 
identify  items  relating  to  the  reliability  engineering  program, 
and  to  verify  that  all  high-risk  reliability  engineering  items, 
as  identified  in  previous  reviews  and  documentation,  are  being 
considered. 
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-  Making  recommendations  as  appropriate  concerning  the 
contractor's  assignment  of  reliability  engineering  specialists 
to  the  review  panel. 

-  Providing  reliability  engineering  support  as  necessary  when 
active  participation  of  the  SPO  in  review  meeting  is  planned. 

-  Evaluating  reliability- rel ated  aspects  of  minutes  to  review 
meetings . 

-  Evaluating  all  Engineering  Change  Proposals  (ECP's)  evolving 
from  the  review  to  assess  the  impact  on  end  item  and  system 
reliability. 

-  Providing  reliability  engineering  guidance  and  support  to  the 
SPO  in  decisions  concerning  approval  of  such  ECP's. 

Specific  activities  of  SPO  reliability  engineering  in  association  with 
specific  PDR,  CDR  and  FACI  programs  are  dependent  on  the  re¬ 
quirements  of  the  particular  development  program  and  on  the  specific 
activities  of  the  contractor  in  preparing  for  and  conducting  the  re¬ 
views.  Typical  objectives  and  activities  of  contractor's  reliability 
engineering  groups  in  preparing  for  and  supporting  these  reviews, 
are  summarized  in  Table  IV- 5.  This  summary,  and  the  more  de¬ 
tailed  discussions  in  the  following  paragraphs  will  provide  guidance 
in  planning  the  activities  of  the  SPO  reliability  engineering  program 
during  the  Acquisition  Phase. 

a.  Preliminary  Review  of  Reliability  Design.  The  first  defined 
a_  ivity  of  the  SPO  Systems  Engineering  following  the  formal 
initiation  of  the  Acquisition  Phase  is  the  support  and/or  par¬ 
ticipation  in  the  Preliminary  Design  Review  (PDR).  This  for¬ 
mal  technical  review  of  the  basic  design  approach  for  each 
CEI  is  usually  performed  by  the  contractor  based  on  review 
criteria  and  agenda  approved  by  the  SPO.  The  responsibility 
of  the  SPO  in  the  PDR  can  vary  from  active  participation  in  the 
review  meetings  to  a  monitoring  action  involving  evaluation  and 
approval  of  the  minutes  of  the  review. 

One  of  the  important  aspects  of  the  PDR  is  the  evaluation  of 
the  engineering  design  approach  to  assure  that  the  CEI  reli¬ 
ability  will  satisfy  the  reliability  requirements  as  established 
by  the  updated  System  Specification  and  the  Part  I  Detail  Speci¬ 
fications.  In  general,  separate  design  reviews  are  not  justified 
for  each  separate  engineering  discipline.  Instead,  reliability 
specialists  are  included  on  the  design  review  panel  and  reliability 
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Table  IV- 5.  Reliability  Engineering  Activities- Acquisition  Phase 


design  is  evaluated  as  an  integral  part  of  the  CEI  design. 
However,  an  analytical  evaluation  of  the  rcdiability  design 
is  usually  performed  prior  to  the  PDR  in  order  to  develop 
the  necessary  inputs  to  the  review.  This  evaluation  will 
normally  be  performed  by  the  contractor.  However,  the 
SPO  can  elect  to  participate  in  order  to  monitor  these  ac¬ 
tivities,  especially  when  high-risk  areas  or  complex  de¬ 
signs  arc  being  evaluated.  Typical  reliability  engineering 
activities  performed  by  the  contractor  for  the  purpose  of 
developing  impacts  for  the  PDR  include: 

Identification  of  reliability  design  factors  that  must 
be  considered  in  establishing  the  agenda  and  schedules 
for  the  PDR. 

Identification  of  high-risk  areas  in  reliability  design 
that  should  be  emphasized  during  the  review. 

Assessment  of  the  level  of  CEI  reliability  being  achieved 
by  the  preliminary  engineering  design.  Typically,  this 
assessment  will  involve  the  performance  of  reliability 
predictions  based  on  updated  reliability  block  diagrams 
and  models  and  reflecting  the  latest  refinements  of  de¬ 
sign  data.  (See  Chapter  9). 

Evaluation  of  reliability  test  program  plans  to  assure 
that  the  latest  revisions  in  systems  and  CEI  reliability 
requirements  are  reflected  in  the  updated  plans  for 
Category  I  and  Category  II  test  programs. 

b.  Critical  Review  of  Reliability  Design.  The  next  formal  activity 
that  is  of  direct  concern  to  the  SPO  reliability  engineering 
activity  is  the  Critical  Design  Review  (CDR),  which  is  per¬ 
formed  at  the  time  the  detailed  design  is  essentially  complete. 
This  formal  technical  review  is  performed  to  evaluate  the  final 
design  prior  to  committing  the  design  to  production.  This  in¬ 
cludes  a  critical  evaluation  of  the  Part  II  Detailed  Specification, 
which  are  the  "build  to"  specifications  that  will  subsequently 
form  the  technical  input  to  the  Product  Configuration  Baseline 
for  production. 

The  reliability  engineering  input  to  the  CDR  follows  essentially 
the  same  general  procedures  as  for  the  PDR's,  but  should  be 
more  critical  in  that  they  are  the  final  review  prior  to  pro¬ 
duction.  Typical  reliability  engineering  activities  that  should 
be  performed  in  performing  this  review,  and  developing  inputs 
for  the  CDR  are: 
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Identification  of  reliability  design  factors  that  should  be 
considered  in  establishing  the  agenda  and  schedules  for 
the  CDR.  This  should  especially  include  those  factors 
that  had  been  identified  as  high-risk  or  problem  areas 
during  the  PDR. 

Identification  of  critical  reliability  design  factors  that 
should  be  given  particular  attention  during  production. 

Assessment  of  the  effectiveness  of  reliability  engineering 
and  related  engineering  design  activities  performed  dur¬ 
ing  the  detail  design  program.  This  includes  the  critical 
review  of  the  results  of  activities  such  as  detailed  reli¬ 
ability  apportionment  analyses,  stress  analysis  reliability 
predictions,  tolerance  and  degradation  analyses,  failure 
modes  and  effects  analyses,  derating,  reliable  parts  selec¬ 
tion,  and  low-level  redundancy  design. 

Assessment  of  the  level  of  CEI  reliability  being  achieved 
by  the  final  design.  This  assessment  should  involve  the 
most  detailed  reliability  analysis,  and,  typically,  would 
include  consideration  of  details  of  design  to  the  part  (or 
equivalent)  level,  and  the  application  of  stres s -factor  reli¬ 
ability  prediction  techniques  using  detailed  part  failure 
rate  data,  such  as  are  presented  in  Volume  II  of  this  note¬ 
book.  In  addition,  this  assessment  should  be  supported  by 
an  evaluation  of  the  results  of  any  test  that  may  have  been 
performed  on  breadboard  or  prototype  models  that  may 
have  provided  data  relating  to  reliability  factors. 

Reliability  Input  to  First  Article  Configuration  Inspection.  A 
third  reliability  engineering  review  point  during  the  Acquisition 
Phase  is  the  activity  necessary  to  support  the  First  Article 
Configuration  Inspection  (FACI).  This  final  formal  design  re¬ 
view  establishes  the  Product  Configuration  Baseline  and  per¬ 
mits  the  formal  acceptance  of  the  Part  II  Detail  Specifications. 

In  general,  the  FACI  concerns  the  comparison  of  the  first 
article  to  be  produced  with  the  verified  requirements  of  the 
Part  II  Detail  Specification,  These  "build  to"  specifications 
do  not  contain  defined  reliability  requirements.  Therefore, 
a  reliability  design  analysis  is  not  required  as  a  direct  input 
to  the  FACI.  In  a  typical  system  development  program,  the 
final  verification  of  design  reliability  is  performed  in  support 
of  the  CDR,  and  as  a  part  of  the  Category  I  and  Category  II 
tests.  In  approving  the  design  for  production,  it  is  assumed 


that  the  required  level  of  reliability,  as  verified  by  the  CDR, 
will  be  achieved  if  the  system  meets  the  requirements  of  the 
Part  II  Detail  Specifications. 

Even  though  specific  reliability  design  and  testing  require¬ 
ments  are  not  included  in  the  Part  II  Detail  Specifications, 
it  is  usually  necessary  to  evaluate  the  design  to  verify  the 
reliability  design  in  areas  that  had  previously  been  identified 
as  high-risk  areas  and  to  assure  that  the  required  level  of 
reliability  is  not  being  degraded  due  to  differences  between 
the  specifications  and  the  hardware  produced,  especially 
where  design  changes  have  been  made  to  facilitate  production, 

6.2  Reliability  Engineering  In  Support  of  Test  and  Evaluation. 

Additional  reliability  engineering  activities  of  the  SPO  during  the 
Acquisition  Phase  include  those  performed  in  support  of  the  Cate¬ 
gory  I  and  Category  II  test  programs.  The  contractor  is  normally 
responsible  for  planning,  implementation,  and  subsequent  follow-up 
activities  in  connection  with  these  test  programs.  Also,  the  SPO 
activities  concerned  with  monitoring  and  controlling  the  Category  I 
and  Category  II  test  programs  are  the  responsibility  of  the  Deputy 
Director  for  Test  and  Deployment,  and  is  not  one  of  the  defined 
engineering  activities.  However,  the  development  of  effective  test 
programs  demands  the  input  of  valid  test  objectives  and  other  factors 
that  are  related  to  the  engineering  design  characteristics  of  the 
system.  Therefore,  extensive  engineering  support  i3  essential  in 
establishing  test  objectives  and  acceptance  criteria,  defining  en¬ 
vironmental  and  operational  test  conditions,  supporting  test  pro¬ 
grams  in  progress,  evaluating  test  data,  and  developing  effective 
follow-up  recommendations. 

The  reliability  engineering  activities  associated  with  the  overall 
engineering  support  of  the  test  program  are  the  responsibility  of 
the  contractor.  To  be  effective,  however,  these  activities  must 
be  fully  monitored,  coordinated,  evaluated  and  controlled  by  the 
SPO.  Therefore,  the  SPO  reliability  engineers  must  support  the 
Deputy  Director  for  Test  and  Deployment  in  areas  outlined  below. 

A  detailed  discussion  of  reliability  testing  is  presented  in  Chapter 

10. 


a.  Verifying  reliability  test  objectives  (e.g.  ,  desired  and  limiting 
MTBF  requirements)  for  each  CEI  to  be  tested. 

b.  Determining  the  validity  and  practicability  of  consumer  risk 
levels  established  for  reliability  testing. 
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c.  Verifying  the  effectiveness  of  reliability  test  plans  in  relation 
to  sampling,  test  duration,  and  ac<  ‘pt /'reject  c  riteria. 

d.  Evaluating  recommended  test  conditions,  including  environ¬ 
mental  and  operational  factors. 

e.  Evaluating  criteria  such  as  definition  of  performance  param¬ 
eters  to  be  measured  during  test,  allowable  degradation  and 
definition  of  failure. 

f.  Monitoring  tests  in  progress  and  resolving  problems  encounter  jd 
that  would  effect  the  validity  of  the  test. 

g.  Reviewing  test  reports  to  evaluate  test  results,  identified,  prob¬ 
lem  areas,  and  recommended  engineering  solutions. 

h.  Advising  the  Program  Director  on  the  extent  of  any  identified 
reliability  engineering  problems,  on  the  selection  of  corrective 
action  approach. 

7.  RELIABILITY  ENGINEERING  MANAGEMENT  DURING  THE 
OPERATIONAL  PHASE 

The  Operational  Phase  of  the  system  life  cycle  begins  with  the  delivery 
and  acceptance  of  the  first  item  on  contract,  and  continues  until  the 
final  disposition  of  the  last  item.  During  this  period,  the  system  op¬ 
eration  becomes  the  responsibility  of  the  using  command  or  activity, 
and  the  system  engineering  responsibility  is  transitioned  from  AFSG 
to  AFLC.  The  change  of  reliability  engineering  responsibility  as  con¬ 
tract  items  are  accepted  by  the  using  command  is  discussed  in  Chapter 
3.  Therefore,  the  specific  reliability  engineering  activities  as  dis¬ 
cussed  here  relate  to  either  AFSC  or  AFLC  engineering  as  appro¬ 
priate. 

Reliability  engineering  during  the  operational  phase  involves  support 
of  operational  reliability  test  programs,  providing  reliability  engineer¬ 
ing'  in  put  for  modifications  design  and  testing  programs,  and  perform¬ 
ing  reliability  engineering  analyses  and  evaluation  of  field  feedback  data. 
These  activities  are  summarized  in  Table  IV- 6,  and  are  discussed  in 
more  detail  in  the  following  paragraphs. 

7.  1  Review  of  Category  III  Test  Program  Requirements. 

This  activity  is  discussed  here  even  though  the  initial  support 
activities  by  the  SPO  usually  begin  before  the  actual  start  of  the 
Operational  Phase.  The  Category  III  tests  are  performed  by  using 
command.  However,  the  SPO  and  system  contractor  provide  en 
gineering  support  in  the  development  of  the  test  program  plans  and 
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Table  IV-6.  Continued 
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procedures.  Such  plans  and  procedures  should  be  reviewed  and 
evaluated  by  reliability  engineering  to  assure  adequate  provisions 
for  reliability  testing. 

The  need  for  and  extent  of  reliability  testing  during  the  Category 
III  test  will  be  determined  based  on  the  requirements  of  the  using 
command,  and  the  operational  requirements  imposed  on  the  sys¬ 
tem.  Also  the  results  of  previous  reliability  engineering  efforts, 
and  of  the  reliability  testing  during  the  Category  I  and  Category  II 
test  programs  should  be  taken  into  consideration  in  developing  the 
reliability  test  procedi  -es  to  be  followed  by  the  using  command. 
Therefore,  an  important  aspect  of  reliability  engineering  is  the 
performance  of  a  critical  review  of  recommended  Category  III 
test  procedures  with  the  objectives  of  assuring  an  appropriate 
and  effective  reliability  test.  This  review  should  include: 

a.  A  review  of  previously  identified  high-risk  areas,  design 
change  recommendations  and  actions,  Category  I  and  Cate¬ 
gory  II  test  reports,  and  other  documentation  to  identify 
specific  needs  for  additional  reliability  testing. 

b.  A  review  of  the  operational  policy  and  objectives  of  the  using 
command. 

c.  A  review  of  other  tests  to  be  performed  during  Category  III 
testing  to  identify  possible  areas  for  reliability  test  consoli¬ 
dation. 

d.  A  review  of  updated  specifications,  diagrams,  and  other  en¬ 
gineering  documentation  to  identify  late  changes  in  reliability 
requirements . 

e.  Consideration  of  time  and  cost  constraints  imposed  on  the 
Category  III  test  programs  and,  from  this,  identification  of 
limitations  on  reliability  testing. 

f.  Review  of  instructions  for  conducting  the  reliability  test,  in¬ 
cluding  sampling  plan,  test  procedures,  and  acceptance  pro¬ 
cedures. 

7.2  Development  of  Foilow-On  System  Modifications. 


Any  follow-on  modifications  that  are  developed  in  response  to  find¬ 
ings  of  Category  III  or  subsequent  operational  test  programs  should 
include  reliability  engineering  as  a  part  of  the  overall  modification 
engineering  effort.  This  should  include  a  reliability  engineering 
effort  with  objectives  and  activities  similar  to  those  of  original 
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systems  engineering  program.  The  extent  of  this  effort,  however, 
is  dependent  on  many  factors,  such  as  the  complexity  of  the  modifi¬ 
cation,  and  the  degree  to  which  the  modification  effects  the  system 
reliability.  ♦ 

7,  3  Define  Follow-On  Reliability  Test  Requirements. 

The  effectiveness  of  follow-on  modifications  must  be  verified  by 
means  of  follow-on  testing  of  the  modified  system.  Such  testing  is 
similar  to  Category  I  or  Category  II  testing  with  the  exception  that 
the  scope  of  testing  is  reduced  to  include  only  those  tests  neces¬ 
sary  tc  verify  the  effectiveness  of  the  modification.  In  general, 
such  tests  will  include  a  minimum  reliability  testing  effort  to  assure 
that  system  reliability  has  not  been  degraded.  However,  at  times 
the  reliability  testing  requires  the  full  support  of  the  SPO  and  con¬ 
tractor  reliability  engineering  groups.  Such  extensive  programs 
may  be  required  in  the  event  of  modifications  to  alleviate  a  serious 
system  reliability  problem. 

7.4  Field  Feedback  Data  Analysis, 

The  analysis  and  evaluation  of  field  feedback  data  is  an  area  of  reli¬ 
ability  engineering  activity  that  begins  with  the  initial  acceptance 
and  operation  of  the  system,  and  continues  as  long  as  there  is  a 
need  for  field  failure  and  operational  data.  The  characteristics 
of  a  field  data  collection  and  reporting  system,  are  discussed  in 
Chapter  7.  Such  a  system  provides  the  basic  data  input  to  the  con¬ 
tinuing  reliability  engineering  efforts  such  as: 

a.  Identification  of  high  failure  rate  equipment  items. 

b.  Determining  operational  and  environmental  causes  of  failure. 

c.  Developing  requirements  for  corrective  action  in  the  event  of 
the  identification  of  latent  reliability  problems. 

d.  Developing  state-of-the-art  reliability  engineering  data  appli¬ 
cable  to  other  system  development  programs. 

8.  RELIABILITY  ENGINEERING  ACTIVITIES  DURING  NON-SYSTEM 
PROCUREMENT  PROGRAMS 

Many  of  the  reliability  engineering  activities  discussed  in  preceding 
paragraphs,  and  most  of  the  techniques  described  in  subsequent  chap 
ters  of  this  notebook  are  applicable  to  individual  equipment  procurement 
programs,  even  though  extensive  development  effort  is  not  required, 
and  the  defined  life  cycle  phases  are  not  identifia.ble.  Quite  often,  for 
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example,  a  "non-system"  program  will  involve  the  normal  develop¬ 
ment  and  procurement  of  new  equipment  items  for  the  operational 
inventory.  In  such  cases,  it  is  possible  to  identify  stages  of  de¬ 
velopment  that  are  similar  to  the  system  life  cycle  phases,  although 
some  system  program  activities  may  not  be  applicable,  and  others 
may  be  contracted  or  deleted  in  the  interest  of  economy  and  expe¬ 
diency.  Other  procurement  programs  are  less  involved,  and  the 
reliability  engineering  a  -tivities  are  appropriately  reduced  in  pro¬ 
portion  to  the  overall  engineering  requirements. 

Tn  general,  non-system  procurement  programs  involve  the  design, 
production  and  testing  of  individual  equipment  items  that  may  or 
may  not  be  part  of  a  defined  system,  In  general,  such  programs 
can  be  classified  as  either  (1)  normal  procurement,  (2)  develop¬ 
ment  and  experimental  model  procurement,  (3)  commercial  equip¬ 
ment  procurement,  or  (4)  low  value  equipment  procurement.  The 
reliability  engineering  activities  associated  with  non-system  pro¬ 
curements  can  vary  considerably,  not  only  with  respect  to  these 
different  classes  of  procurement,  but  also  with  respect  to  individual 
procurement  programs  within  a  given  cl^.ss.  However,  certain 
basic  reliability  engineering  activities  are  applicable  to  any  pro¬ 
curement.  Table  IV-7  lists  the  more  important  reliability  engineer¬ 
ing  activities  normally  associated  with  each  general  class  of  non¬ 
system  procurement.  The  chapter  references  in  the  right  hand  col¬ 
umn  refer  to  subsequent  chapters  of  this  notebook  that  contain  infor¬ 
mation  relating  to  the  particular  activities.  This  list  should  not  be 
considered  as  all-inclusive  but  can  be  used  as  a  guide  in  establish¬ 
ing  reliability  engineering  requirements  for  non-system  programs. 
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Table  IV-7.  Reliability  Engineering  Activities  During  Non-System  Programs 
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Analyze  reliability  test  and  field 
data  and  develop  design  changes 
for  reliability  improvement. _ 


CHAPTER  5 


RELIABILITY  PROGRAM  BATA 

AIR  FORCE  SYSTEM  DEVELOPMENT  DATA  REQUIREMENTS 

The  formal  methodology  instituted  by  the  Air  Force  for  managing 
complex  system  development  programs  includes  five  major  areas 
of  management  activity:  procurement  and  production;  program 
control;  configuration  control;  system  e  lgineering;  and  test  and 
deployment.  Effective  coordination  of  all  areas  of  management  re¬ 
quires  a  unifi.  rrn  system  for  controlling  the  acquisition  and  distri¬ 
bution  ol  management,  scientific,  engineering,  and  logistics  infor¬ 
mation,  reports  and  documentation.  Such  "data",  which  are  es¬ 
sential  to  the  management  of  a  system  development  program,  are 
managed  in  accordance  with  a  standardized  procedure  as  required 
byAFR310-l,  and  implemented  by  AFSCM/  AFLCM  310-1. 

Air  Force  Regulation  AFR  3  1  0- i  establishes  official  Air  Force  policy 
concerning  the  identification,  justification,  selection,  acquisition,  and 
control  of  all  data  relevant  to  system  development. 

In  response  to  the  requirements  of  AFR  310-1,  AFSC  requires  details 
of  system  development  programs  to  be  documented  and  reported  by 
means  of  a  strictly  controlled  series  of  recurring  and  non-recurring 
reports  and  other  forms  of  documentation.  This  data  acquisition  pro¬ 
gram  is  managed  in  accordance  with  AFSCM/ AFLCM  3 1 0- 1,  "Manage¬ 
ment  of  Contractor  Data  and  Reports,  "  which  prescribes  data  manage¬ 
ment  procedures,  and  includes  the  Authorized  Data  List  (Volume  II  of 
AFSCM/AFLCM  3 1  0- 1 ) .  This  list  specifies  the  content  and  format  of 
approximately  340  standardized  "Data  Items"  that  have  been  approved 
for  use  on  AFSC  development  contracts.  These  Data  Items  have  some 
application  during  essentially  all  phases  of  the  system  life  cycle,  but 
are  primarily  concerned  with  the  definition  and  acquisition  phases  when 
contractor  activities  are  most  significant. 

1.  1  Reliability  Data  Requirements  in  System  Development  Programs 

Many  of  the  Data  Items  specified  in  AFSCM/AFL CM.  3 1 0- 1  are  in¬ 
tended  for  the  direct  -'upport  of  the  reliability  program.  Other 
Data  Items,  which  are  intended  to  support  other  activities  of  the 
system  development  program,  also  require  certain  types  of  re¬ 
liability  data  that  are  essential  to  other  aspects  of  the  system  de¬ 
velopment  program. 

The  Authorized  Data  List  is  universal  in  scope,  and  contains  data 
iterrls  applicable  to  a  variety  of  development  programs.  Therefore, 
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a  certain  degree  of  selectivity  is  essential  in  establishing  re¬ 
liability  data  requirements  for  specific  system  development 
programs.  It  is,  therefore,  the  intent  of  this  chapter  to  pro¬ 
vide  a  guide  for  the  selection  of  reliability  data  requirements 
for  application  to  a  system  development  program,  and  for  the 
management  of  these  data  during  the  conceptual,  definition  and 
acquisition  phases  of  the  system  life  cycle.  This  includes  con¬ 
sideration  of  certain  data  items  not  specifically  identified  in  the 
"reliability"  category,  but  which  require  specific  reliability  data 
inputs,  or  which  may  otherwise  impact  on  the  reliability  program. 

1.2  Reliability  Data  Requirements  of  Non-System  Programs 

The  data  management  procedures  specified  in  AFSCM/AFLCM  3  1  0- 1 
are  based  on  the  requirements  of  system  development  programs 
that  are  managed  in  accordance  with  AFSCM  375-4.  However, 
many  of  the  data  items  in  the  Authorized  Data  List  are  also  appli¬ 
cable  to  the  procurement  of  individual  equipment  items,  experi¬ 
mental  or  developmental  models,  commercial  items,  or  other 
types  of  procurement  not  warranting  a  formal  system  program 
approach.  For  example,  a  program  for  the  procurement  of  an 
individual  equipment,  and  which  involves  development  and  design 
engineering  and  analysis  tasks,  will  require  essentially  the  same 
types  of  reliability  data  as  a  system  development  program,  even 
though  the  AFSCM  375  series  documents  are  not  imposed.  The 
extent  of  the  data  requirements  for  non- system  programs  will  vary 
depending  on  the  type  of  procurement,  the  amount  of  engineering 
required,  and  the  importance  of  reliability  in  relation  to  the  in¬ 
tended  use  of  the  equipment.  Even  the  least  demanding  programs 
will  require  reliability  specification  and  acceptance  test  require¬ 
ments  data  to  assure  the  procurement  of  appropriately  reliable 
items . 

In  most  cases  the  reliability  program  data  can  be  obtained  by 
selective  application  of  data  items  from  the  Authorized  Data  List. 
However,  due  to  the  economic  considerations  associated  with  most 
non-system  procurements,  the  reliability  data  requirements  should 
be  limited  to  those  that  are  essential  to  the  particular  program.  In 
view  of  this,  a  brief  discussion  and  guide  for  the  selection  and  appli¬ 
cation  of  AFSCM/AFLCM  3  1 0- 1  data  items  to  a  variety  of  non- sys¬ 
tem  procurement  programs  is  included  in  the  latter  portions  of  this 
chapter. 
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2.  RELIABILITY  DATA  REQUIREMENTS  IN  AFSCM/AFLCM  310-1 


The  Authorized  Data  List  (Volume  II  of  AFSCM/AFLCM  310-1)  contains 
the  approved  AFSC/AFLC  Form  9,  or  "Data  Items,  "  which  are  authorized 
for  possible  use  on  system  development  contracts.  Each  Data  Item  has 
been  assigned  to  one  of  thirteen  functional  categories  depending  on  the 
most  common  use  of  the  particular  data. 

The  functional  categories  that  are  of  major  interest  to  reliability  pro¬ 
gram  management  are: 

Category  C  -  Configuration  Management 

Category  R  -  Reliability/Maintainability 

Category  S  -  Sy  stem /Subsystem  Analysis 

Category  T  -  Test 

Other  functional  categories  also  contain  date,  items  that  are  indirectly 
related  to  the  reliability  program.  However,  any  such  reliability  data 
requirements  are  generally  derived  from  the  data  included  in  the  four 
categories  mentioned  above.  Therefore^appropriate  management  of 
these  major  data  items  will  assure  the  Acquisition  of  essential  reliability 
data  for  use  in  connection  with  other  areas  of  management. 

Category  R  contains  the  Reliability /Maintainability  Data  Items  and, 
therefore,  contains  those  data  items  that  most  directly  concern  the  reli¬ 
ability  program.  Category  R  includes  fifteen  data  items.  Eight  of  these 
are  specifically  related  to  activities  of  the  reliability  program,  while  two 
others,  which  are  more  general  in  nature,  concern  both  the  reliability  and 
maintainability  program.  The  relationship  of  these  data  items  to  the  reli¬ 
ability  program  is  apparent. 

The  remaining  four  Category  R  data  items:  which  are  specifically  addressed 
to  the  activities  of  the  maintainability  program,  also  contain  requirements 
that  directly  or  indirectly  impact  on  the  reliability  program.  For  example, 
certain  reliability  data  are  essential  in  developing  inputs  to  the  maintainab¬ 
ility  data  items.  Therefore,  in  selecting  reliability  data  items  for  applica¬ 
tion  to  a  particular  contract,  consideration  should  also  be  given  to  the  data 
requirements  of  the  maintainability  program. 

The  specific  reliability  data  requirements  of  data  items  list  in  Functional 
Categories  C.  R,  S  and  T  are  summarized  in  Table  V-l.  This  table  ident¬ 
ifies  specific  data  items  by  number  and  title,  and  briefly  summarizes  the 
reliability  data  requirements  of  each.  Unless  otherwise  indicated,  the 
references  in  the  right-hand  column  refer  to  specuic  paragraphs  of  the  data 
item  under  consideration. 
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Proposal  -  Reconnaissance  Subsystem  -  Predicted  reliability  of  proposed  Paragraph  (lb) 

design 

-  Data  source  and  analytical  approach 

used  in  prediction 

-  Proposed  reliability  test  program 


Functional  Category  S  -  System/Subsystem  Analysis  (Continued) 
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Category  I  or  II  testing  is  not  appli¬ 
cable) 


3.  RELIABILITY  DATA  REQUIREMENTS  CITED  IN  CONTRACTS 
DURING  THE  SYSTEM  LIFE  CYCLE 

The  reliability  data  requirements  as  specified  in  AFSCM/AFLCM  3  1  0- 1 
are  used  as  required  throughout  the  Definition  and  Acquisition  phase  of 
system  development  and  during  equivalent  stages  of  non-system  develop¬ 
ment  (see  Chapters  3  and  4). 

Certain  reliability  data  are  also  obtained  during  the  Development  and 
Transition  stages  of  Lhe  Conceptual  phase  before  the  SPO  is  established. 
Some  of  these  data  can  be  obtained  via  standardized  data  items.  How¬ 
ever,  most  of  the  reliability  data  generated  during  the  Conceptual  phase 
are  applied  to  the  development  of  the  PT  DP,  and  are  developed  bv  Air 
Force  rather  than  contractor  activities. 

Additional  reliability  data  are  obtained  during  the  Operational  phase  of 
the  system  life  cycle.  However,  most  of  these  data  are  in  the  form  of 
empirical  operational  and  failure  data,  and  are  obtained  via  the  Air 
Force  Maintenance  Data  Collection  System  as  discussed  in  Chapter  7. 

Reliability  data  requirements  commonly  encountered  during  the  Con¬ 
ceptual,  Definition,  and  Acquisition  phases,  and  which  are  obtained 
via  the  standardized  Data  Items  of  AFSCM/AFLCM  3 1 0- 1,  are  sum¬ 
marized  by  phase  in  Table  V-Z.  Additional  reliability  program  data 
generated  during  the  Conceptual  phase  are  reviewed  in  Paragraph  4 
of  this  chapter. 

4.  AIR  FORCE  -  GENERATED  RELIABILITY  DATA 

In  general,  reliability  data  acquired  from  contractors  during  the  system 
life  cycle  will  be  limited  to  the  type  indicated  in  Paragraph  3.  However, 
before  these  data  can  be  developed,  certain  reliability  data  must  be  gen¬ 
erated  by  the  Air  Force.  Such  data  are  typically  developed  during  the 
conceptual  phase  in  preparing  for  Transition  to  the  Definition  Phase 
activities,  and  during  Phase  A  of  the  Definition  Phase.  Thereafter, 
reliability  data  development  will  become  the  responsibility  of  the  con¬ 
tractor. 

Based  on  knowledge  gained  from  the  early  development  and  system  study 
activities  of  the  conceptual  phase,  Headquarters  USAF  issues  the  SOR/ 
OSR/ADO  documents  which  establish  fundamental  system  requirements. 
These  documents  provide  the  basis  for  the  subsequent  development  of 
more  detailed  reliability  data.  Typical  information  from  which  subsequent 
reliability  data  are  developed  include: 
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(1)  Basic  system  functional  structure  and  time  relationships. 


(2)  Basic  reliability  and  maintainability  allocations. 

(3)  Reliability  block  diagrams. 

(4)  Estimates  of  allowable  in-commission  rates,  downtime  allocations, 
manpower,  and  available  maintenance  resources. 

4.  1  Reliability-Related  Input  to  the  PTDP 

In  response  to  the  SOR/OSR/ADO,  the  Air  Force  project  manage¬ 
ment  develops  the  Preliminary  Technical  Development  Plan  (PTDP), 
which  defines  the  technical  portion  of  the  Program  Requirements 
Baseline  governing  the  activities  of  the  Definition  Phase.  The  re¬ 
liability  data,  and  supporting  information  included  in  the  PTDP  will 
provide  the  reliability  requirements  for  the  initial  system  specifi¬ 
cation.  Therefore,  in  the  development  of  the  PTDP,  the  Air  Force 
should  assure  that  the  following  types  of  information  and  data  are 
included: 

(1)  Planned  system  functional  description,  including: 

Functional  diagrams 

Engineering  descriptions  of  the  functions 

Established  system  design  requirements 

Gross  solutions  to  these  requirements 

Predicted  equipment  configuration 

Trade-offs  considered  and  areas  requiring  further 
exploration  (high  risk  technical,  cost,  or  schedule 
areas ) 

(2)  Preliminary  operation  plans,  including: 

Mission  duration  requirement  for  each  type  of  mission 
Reaction  time,  availability,  and  ready  rates  required 
Planned  utilization  rates  of  system  elements 
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(3)  Plans  for  a  reliability  program  outlining  how  reliability 
requirements  will  be  achieved,  providing  for- 

Overall  mission  reliability  for  each  type  of  mission 

Reliability  after  storage  goals;  other  measures  as 
required 

Reliability  apportionment,  prediction,  and  modeling 

Determination  of  equipment  environmental  conditions 

Periodic  specification  review  (when,  how  often,  etc.  ) 

Reliability  participation  in  System  Engineering  Design 
Reviews 

Coordination  with  human  error  analysis  and  prediction 

Reliability  tests,  demonstration,  and  resolution  of  problem 
areas 

Malfunction  and  failure  reporting  and  analysis 

Planned  products,  milestones,  and  schedules 

4.  2  Reliability  Input  for  the  Initial  General  Specification 

The  most  significant  reliability  data  to  be  developed  by  the  3PO 
during  Phase  A  of  the  Definition  Phase  is  the  reliability  require¬ 
ment  and  corresponding  reliability  assurance  provisions  for  the 
initial  System  Performance /Design  Requirements  General  Specifi¬ 
cation.  This  specification  is  developed  following  Exhibit  I  of 
AFSCM  375-  1.  (Subsequent  generations  of  the  General  Specification 
will  be  developed  by  the  contractor  in  accordance  with  Data  Item 
C-l-35.  1-1  of  AFSCM /AFSLM  310-1,  as  discussed  in  Paragraph  3.) 
The  system  reliability  data  included  in  the  General  Specification 
includes: 

(1)  System  reliability  requirements  stated  in  quantitative  terms 

(2)  Conditions  under  which  requirements  are  to  be  met 

(3)  Reliability  apportionment  model  (when  appropriate)  allocating 
reliability  to  system  segments 


(4)  Reliability  testing  requirements  at  various  levels  of  system 
assembly 

(5)  Reliability  test  data  recording  requirements  as  applicable 
during  Category  I  and  II  testing. 

5.  RELIABILITY  DATA  REQUIREMENTS  FOR  NON-SYSTEM  PROGRAMS 

The  reliability  data  requirements,  as  discussed  in  preceding  paragraphs, 
are  based  on  specific  Data  Items  listed  in  AFS CM /AFLCM  3 1  0 -  1 ,  and  on 
certain  configuration  management  requirements  of  AFSCM  375-  1.  These 
documents  are  based  on  large  system  acquisition  programs  wherein  specific 
life  cycle  phases  can  be  defined.  Quite  often,  the  data  requirements  of 
non-system  programs  are  similar  to  those  of  a  system  program,  even 
though  specific  program  phases  may  not  be  defined.  However,  the  degree 
of  detail  and  complexity  of  data  requirements  applicable  to  large  systems 
is  no,  always  necessary  in  many  small  scale  programs.  For  example, 
procurement  of  individual  "off-the-shelf"  equipments,  and  other  such 
equipment  items,  may  not  warrant  a.  full-scale  data  management  pro¬ 
gram.  This  is  true  of  reliability  data  as  well  as  other  categories  of  data 
normally  obtained  during  system  development. 

5.  1  Typical  Nor.-System  Reliability  Data  Requirements 

Even  though  extensive  reliability  data  are  not  always  required  on 
such  programs,  there  is  still  a  need  for  a  minimum  amount  of  data 
to  specify  a  reliability  requirement,  and  to  assure  that  appropriately 
reliable  items  are  being  procurred.  Typical  reliability  data  require¬ 
ments  for  several  categories  of  non- system  procurements  are  des¬ 
cribed  below. 

a.  Normal  P rocurements .  Service  Test,  Preproduction  and  Develop¬ 
ment  Models,  and  individual  equipments  procurred  for  operational 
use. 

(1)  Mean  Time  Between  Failures  (MTBF),  requirements  and 
failure  definition 

(2)  Useful  Life  Requirements 

(3)  Test  Requirements  -  Reliability  Demonstration 

Definition  of  test  plan  and  test  level  and  applicable 
MIL  Specification 

Justification  for  chosen  test  plan  and  *est  level 

Test  time  and  definition  of  allowable  failures  and 
level  of  confidence 
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(4)  Production  reliability  test  plan  (when;  applicable] 

b.  Advanced  Development  and  Experimental  Models  not  Procurred 
for  Operational  Use. 

(1)  Functional  diagrams 

(2)  Parts  count 

(3)  Description  of  stress  analysis 

(4)  Description  ol  modeling  tools 

(5)  Computed  reliability  parameters 

(6)  Reliability  den’  .nstration  requirements 

c.  Commercial  "Off-the-shelf"  Equipment. 

(1)  Mean  time  between  failure  requirements,  and  failure 
definition 

(2)  Reliability  assessment  report 

(3)  Historical  reliability  data 

d.  Low  Value  Items  for  Non-Critical  Use, 

(1)  MTBF  requirements  and  failure  definition 

(2)  Reliability  demonstration  requirements; 

Test  Plan 
Test  T ime 

Maximum  allowable  failures 

5.2  AFSCM/A FLC  M  3  1  0- 1  Data  Items  Applicable  to  Non-System  Procure¬ 
ments 

Many  of  the  data  items  identified  and  described  in  Paragraph  2  of  this 
chapter  are  applicable  for  acquiring  reliability  data  for  many  non¬ 
system  procurement  programs.  In  general,  the  particular  data  items 
used  must  be  selected  accordmg  to  the  particular  requirements  of 
the  program  under  consideration.  In  meet  cases,  however,  the 
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reliability  data  requirements  will  be  satisfied  by  selecting  data 
items  as  indicated  in  Table  V-3.  In  any  case,  the  specific  data 
items  selected  should  be  reviewed  and  revised  where  necessary 
to  assure  that  all  essential  data  are  being  obtained,  and  to  delete 
any  unnecessary  data  requirements. 
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Table  V-3  Data  Items  Applicable  to  Non-System  Procurement  Programs 


Data  Item 

Numbe  r 

Normal 

Procurement 

Development 

Experimental 

Commercial 

"Off-Shelf'' 

Low 

Value 

R-l  01 

/ 

R  - 1  02 

! 

R-l  03 

/  * 

/  * 

R-l  04 

/ 

J 

/ 

R-l  05 

J 

R-l  06 

J 

J 

/ 

R-l  07 

J 

R-108 

/ 

R-l  09 

J  * 

R-110 

J 

R-l  1 1 

J 

R-l  1  2 

J 

/ 

V 

R- 113 

J 

R-l  1  4 

/ 

R-l  15 

J 

*  Applicable  only  in  procurement  of  large,  complex  equipment  items. 
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CHAPTER  6 


ASSURING  RELIABILITY  PROGRAM  EFFECTIVENESS 


1.  INTRODUCTION 

Reliability  requirements  have  become  a  major  clement  in  planning, 
developing,  designing,  engineering,  producing  and  purchasing  military 
equipments  and  systems.  As  with  other  performance  characteristics, 
such  as  range  and  power  output,  for  which  requirements  are  established, 
it  is  important  to  recognize  those  efforts  and  activities  which  must  be 
undertaken  before  the  final  product  is  tested  and  delivered  in  order  to 
have  some  assurance  that  performance  objectives  will  be  met.  For  in¬ 
stance,  before  hardware  is  fabricated  it  is  normal  to  spend  time  and 
effort  (often  considerable  time  and  effort)  in  the  "paper  design"  stage, 
during  which  extensive  calculations  are  made  to  "show"  that  the  design 
(which  includes  many  factors,  such  as  components,  structural  relation¬ 
ships,  and  expected  values)  will  meet  required  performance  objectives. 
Stages  which  follow  the  paper  design  are  usually  undertaken  only  after 
there  is  some  assurance  (either  through  calculations,  review,  com¬ 
parisons,  simulation  or  some  combination  of  all  of  these)  that  the  de¬ 
sign  will  meet  required  performance  objectives.  Each  progressive 
stage  through  complete  fabrication  is  undertaken  only  after  results 
have  been  developed  which  provide  a  reasonable  degree  of  assurance 
that  the  outcome  will  be  successful.  Finally  the  product  (i.  e.  ,  com¬ 
ponent,  equipment,  or  system)  will  be  produced  and  delivered  only 
after  tests  show  that  objectives  or  requirements  for  performance 
characteristics  will  be  met. 

This  same  general  pattern  of  programmed  or  planned  activities  such  as 
review  through  test  is  also  appropriate  for  providing  assurance  that  the 
final  product  will  be  delivered  with  the  required  reliability.  That  is, 
it  is  appropriate  to  think  in  terms  of  a  reliability  program  as  a  means 
of  ensuxing  that  reliability  requirements  '  ixbjectives  will  be  met.  The 
purpose  of  the  reliability  program  would  be  to;  (1)  plan  for  the  appro¬ 
priate  reliability  oriented  activities,  including  establishing  reliability 
requirements;  (2)  ensure  that  these  activities  are  being  carried  out; 

(3)  monitor  and  evaluate  results;  (4)  ensure  that  there  are  sufficient  con¬ 
trols  to  prevent  implementing  succeeding  stages  without  having  successfully 
completed  preceding  stages;  and  (5)  ensure  that  meaningful  criteria  have 
been  established  and  are  being  used  as  che  basis  for  decisions  concerning 
acceptability  or  non-acceptability  of  results. 
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2.  RELIABILITY  PROGRAM  RESPONSIBILITIES 


In  general,  the  assurance  of  meeting  reliability  requirements  and  assuring 
reliability  program  effectiveness  is  a  responsibility  which  must  be  under¬ 
taken  by  the  System  Project  Office  (SPO)  and  the  hardware  contractor. 

Contractor  responsibility  is  oriented  toward  responding  to  the  quantitative 
reliability  requirements  and  providing  the  organization  and  functional  re¬ 
sources  and  control  which  will  provide  the  assurance  that  the  reliability 
program  will  be  effective  and  that  reliability  objectives  will  be  met. 

From  the  Slystem  Project  Office  (SPO)  or  program  manager  point  of  view, 
there  are  three  areas  of  general  concern.  First,  there  is  the  concern  over 
the  reliability  requirements  as  a  performance  characteristic.  That  is,  the 
SPO  or  project  manager  responsibility  includes  the  requirement  to  know 
what  is  desired  in  the  way  of  reliability  requirements  and  to  express  these 
in  a  meaningful  way  to  hardware  contractors.  Second,  there  is  concern 
over  what  constitutes  an  effective  reliability  program  consistent  with 
established  quantitative  requirements  as  a  performance  characteristic. 

That  is,  what  are  the  elements  of  a  sound  reliability  program,  including 
contractor  organizational  structure  and  levels  of  responsibility.  Third, 
there  is  concern  over  the  methods  or  techniques  which  should  be  used  to 
assure  the  effectiveness  of  a  reliability  program.  That  is,  what  are  the 
criteria  for  acceptance  and  what  are  the  means  for  conducting  effective 
monitoring  and  evalxiation  of  performance. 

Major  elements  impacting  on  reliability  program  effectiveness  can  be 
described  as  falling  into  five  areas:  (1)  reliability  specification;  (2)  planned 
reliability  activities,  such  as  design  review  and  testing;  (3)  evaluation  of 
reliability  plans  and  organizational  structure;  (4)  preparation  of  criteria 
to  be  used  as  a  basis  for  conducting  evaluation;  and  (5)  establishing  com¬ 
munication  links  to  ensure  rapid  response  both  to  problem  areas  and  to 
acceptable  results. 

Air  Force  documents  provide  the  SPO  and  project  managers  with  the 
means  for  imposing  general  reliability  program  requirements  and  acquir¬ 
ing  appropriate  information  from  the  contractor.  In  particular,  AFSCM/ 
AI'LCM  310-1  provides  the  basis  for  imposing  reliability  program  and 
documentation  requirements  and  the  AFSCM  375  series  provides  useful 
information  concerning  definitions  and  general  guidelines.  The  purpose 
of  this  chapter  is  to  provide  an  overview  of  the  information  contained  in 
these  documents  and  to  provide  information  which  would  lead  to  more 
elective  utilization  of  means  for  assuring  reliability  program  effective¬ 
ness. 
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3.  SPECIFYING  RELIABILITY  REQUIREMENTS 


Specifying  reliability  requirements  is  one  of  the  key  SPO  responsibilities 
and  probably  one  of  the  first  elements  which  should  be  considered  in 
establishing  the  need  for  a  reliability  program.  Reliability  requirements 
generally  consist  of  two  distinct  segments.  First  there  is  the  segment  of 
the  reliability  requirement  which  is  performance  oriented.  It  is  extremely 
important  that  this  segment  of  the  reliability  requirement  be  expressed 
quantitatively  in  a  manner  (i.  e.  ,  using  the  reliability  definition,  measure, 
or  unit)  which  is  consistent  with  mission  or  tactical  objectives.  Second, 
there  is  the  segment  of  the  reliability  requirement  which  is  activity,  data, 
test,  or  "program"  oriented.  This  segment  of  the  requirement  also  must 
be  quantitative  where  possible  (the  test  requirement,  for  example)  and  it 
must  impose  only  those  activities  and  data  deliverables  which  are  con¬ 
sistent  with  the  type  hardware  program  and  the  quantitative  reliability  re¬ 
quirements. 

From  AFSCM  375-3,  (15  June  1964  issue,  paragraph  33,  page  44)  reli¬ 
ability  is  defined  as  the  probability  that  a  system  will  perform  its  desig¬ 
nated  mission  for  the  specified  length  of  time  in  the  operational  environ¬ 
ment.  The  resultant  reliability  in  its  broadest  sense  is  a  measure  of  how 
well  everyone  has  done  his  job.  It  is  the  result  of  many  interplaying  fac¬ 
tors,  many  of  which  are  engineering  responsibilities  (see  AFSCR.  80-1). 
This  is  a  standard  definition;  however,  there  are  other  measures  of  reli¬ 
ability  such  as  Mean  Time  Between  Failure  (MTBF),  Failure  Rate,  and 
Mean  Time  Before  Maintenance  (MTBM).  These  latter  reliability  mea¬ 
sures  are  often  more  directly  useable  as  design  criterion  and  therefore 
the  application  of  these  measures  is  often  the  basis  for  communication 
between  engineers.  In  any  event,  the  SPO  or  project  manager  has  the 
responsibility  for  ensuring  that  the  reliability  requirement  is  expressed 
in  quantitative  terms.  Policy  concerning  quantitative  reliability  require¬ 
ments  is  established  in  AFR  80-5  and  AFSCR  80-1.  Quantitative  reli¬ 
ability  requirements  provide  performance  parameters  which  must  be 
specified,  designed  to,  and  measured.  This  quantification  process  is 
basic  to  achieving  the  reliability  desired  in  modern  systems. 

In  addition,  the  SPO  or  project  manager  has  the  responsibility  for  im¬ 
posing  reliability  program  requirements  and  test  requirements.  MIL-R- 
27542  (Reliability  Program  for  System?,  Subsystems,  and  Equipment) 
establishes  requirements  for  a  con.;, i  uhensive  reliability  program.  This 
specification  is  suitable  as  a  contractual  call-out  for  full  compliance. 
However,  if  che  peculiarities  of  a  program  indicate  that  deviations  would 
be  in  the  best  interests  of  the  Air  Force,  such  deviations  should  be  care¬ 
fully  defined  and  negotiated  with  the  contractor  prior  to  entering  into  an 
Acquisition  Phase  contract.  MIL-R-27542  elaborates  on  various  elements 
of  a  reliability  program  such  as  reliability  requirements,  reliability 


apportionment,  reliability  estimates,  design  reviews,  failure  data  col¬ 
lection,  analysis,  corrective  action,  and  reliability  demonstrations  on 
or  before  specified  program  milestones. 

In  establishing  tc  st  requirements,  for  example,  experience  has  shown 
that  for  systems  with  very  high  reliability  requirements,  a  production 
reliability  testing  program  is  essential.  Such  a  program  normally  pro¬ 
vides  random  selection  of  production- run  equipment  for  specific  reli¬ 
ability  testing.  In  some  cases  complete  testing  may  be  justified,  but 
generally  equipment  will  be  tested  for  critical  weakness,  electromag- 
netic  interference  effects,  margins  of  safety,  and  reliability  data. 

As  mentioned  above,  reliability  requirements  and  particularly  reliability 
performance  requirements  should  be  quantitatively  expressed  whenever 
possible  and  the  unit  of  measure  used  should  be  consistent  with  intended 
use  and  operational/ tactical  objectives.  For  instance,  the  units  such  as 
"probability  of  survival  for  a  given  mission  time"  may  not  be  appropriate 
for  a  communications  system  which  is  expected  to  operate  continuously. 

In  the  case  of  a  communications  system  the  probability  that  a  given  num¬ 
ber  of  channels  is  "up"  at  any  one  time  or  a  mean  time  between  failure 
(in  hours)  for  each  channel  may  be  more  meaningful.  In  addition  a  reli¬ 
ability  requirement  for  meeting  a  given  probability  of  mission  completion 
should  include  confidence  bounds  in  order  to  provide  a  basis  for  a  meaning¬ 
ful  evaluation.  A  reliability  requirement  in  the  form  of  a  probability  of 
mission  completion  with  appropriate  confidence  bounds  is  more  meaningful 
and  results  in  a  more  complete  reliability  statement  based  on  test  or 
demonstration  data.  It  should  be  noted,  however,  that  before  a  confidence 
bounds  statement  (i.  e.  ,  requirement)  can  be  established  or  developed 
from  data,  equipment/system  failure  characteristics  or  distribution  (e.  g.  , 
exponential,  Weibull,  gamma)  must  be  known  or  assumed.  An  erroneous 
assumption  concerning  the  appropriate  distribution  could  lead  to  a  wide 
disparity  between  expected  and  actual  reliability  performance  on  an  equip¬ 
ment  or  system  which  passed  acceptance  criteria  based  on  an  assumed 
failure  characteristic.  See  Chapter  10,  Reliability  Measurement  for  a 
more  comprehensive  presentation  on  measuring  reliability. 

Alternative  mission  requirements  and  the  correspondingly  most  appropriate 
means  for  expressing  reliability  together  with  the  underlying  technical  de¬ 
tails  related  to  various  possible  failure  characteristics  or  distributions 
are  far  too  lengthy  to  be  included  here.  However,  there  are  guidelines 
which  the  SPO  or  project  manager  can  follow  in  determining  how  the 
quantitative  reliability  performance  requirement  should  be  expressed. 
Guidelines  which  are  presented  in  Table  6-1  are  based  on  the  following 
four  basic  ways  in  which  a  reliability  requirement  can  be  expressed. 


(1)  As  a  mean  Aime-between-failure,  MTBF.  This  definition  is  useful 
for  long-life  systems  in  which  the  form  of  the  reliability  distribution 
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is  not  too  critical,  or  where  the  planned  mission  lengths  are  always 
short  relative  to  the  specified  mean  life.  This  definition  is  appro¬ 
priate  for  specifying  mean  life,  however,  equipment  behavior  in 
early  life  may  not  be  the  specified  level  of  reliability,  beciuse  early 
failures  due  to  design  "bugs "may  initially  distort  he  true  sys tern  behavior. 

(2)  As  a  probability  of  survival  for  a  specified  period  of  time,  t.  This 
definition  is  useful  for  defining  reliability  when  a  high  reliability  is 
required  during  the  mission  period,  but  mean-time-to-failure  beyond 
the  mission  period  is  of  little  tactical  consequence  except  as  it  in¬ 
fluences  availability, 

(3)  As  a  probability  of  success,  independent  of  time.  This  definition  is 
useful  for  specifying  the  reliability  of  one-shot  devices  and  those 
which  are  cyclic,  such  as  the  flight  reliability  of  missiles,  the  launch 
reliability  of  launchers,  or  the  detonation  reliability  of  warheads. 

(4)  As  a  "failure  rate"  over  a  specified  period  of  time.  This  definition 
is  useful  for  specifying  the  reliability  of  parts,  components,  and 
modules  whose  mean  lives  are  too  long  to  be  meaningful,  or  whose 
reliability  for  the  time  period  of  interest  approaches  unity. 

4.  RELIABILITY  DESIGN  REVIEWS 

Once  a  reliability  requirement  has  been  developed,  some  means  must  be 
established  for  monitoring  progress  toward  meeting  this  requirement  and 
for  monitoring  contractor  performance  to  ensure  that  reliability  oriented 
activities  as  required  and  proposed  have  been  implemented  and  are  being 
effectively  carried  out.  One  means  of  accomplishing  this  monitoring  is 
through  formally  held  and/or  documented  design  reviews  either  as  part  of 
engineering  reviews  which  include  reliability  or  through  reviews  which 
are  held  specifically  for  reliability. 

In  general,  engineering  design  reviews  and  evaluations  should  include  reli¬ 
ability  as  a  tangible  operational  characteristic  of  the  system,  equipment, 
assembly,  or  circuit  under  review.  Reliability  considerations  during  the 
design  reviews  should  include: 

(1)  Performance  requirements  and  definitions  of  failure  (e.  g.  ,  tolerances, 
wear,  and  parameter  shifts). 

(2)  The  apportionment/modeling  techniques  used  to  establish  the  reli¬ 
ability  design  goals  for  sub-units  which  will  ensure  that  the  reli¬ 
ability  specified  for  units  will  be  met. 
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(3)  A  reliability  prediction  of  the  current  design,  supported  by  detailed 
calculations  and  data  sources. 

(4)  Failure  mode  analysis  of  the  design  with  particular  emphasis  upon 
the  reduction  of  marginal  failure  modes  which  are  difficult  to  isolate 
and  repair. 

(5)  Evaluation  of  tra.de-offs  between  performance,  maintainability,  weight, 
space,  power,  cost,  and  time  factors  made  for  design  optimization. 

(6)  Environments  to  which  the  device,  item,  or  circuits  will  be  subjected 
in  the  use  configuration,,  including  storage,  transport,  and  production 
process  environments. 

(7)  Planned  tests  and/or  the  results  of  all  tests  conducted  to  date. 

(8)  Plans  for  reliability  improvement  and  problem  solutions. 

In  addition,  reviews  should  include  an  evaluation  of  the  contractor's  reli¬ 
ability  program  from  a  functional/organizational,  reporting,  control, 
and  task  point  of  view.  That  is,  the  review  should  be  used  as  a  means  for 
monitoring  reliability  "progress"  with  re’  ect  to  the  complete  reliability 
requirement,  quantitative  as  well  as  qualitative.  Thus  a  description  of 
the  organization  and  personnel  responsible  for  reliability,  the  tasks  being 
conducted  or  completed,  and  the  repor ting/ sign-off  structure  would  pro¬ 
vide  an  indication  of  the  degree  to  which  reliability  is  being  monitored  and 
controlled  within  the  contractor’s  organization.  This  process  of  evaluating 
reliability  design  progress  as  well  as  reliability  program  implementation 
could  be  a  very  useful  means  of  assuring  reliability  program  effectiveness. 

The  detail  to  which  early  reliability  design  review  is  conducted,  whether  as 
a  separate  effort  or  as  part  of  regular  engineering  design  reviews  is  a  func¬ 
tion  of  the  point  in  the  cycle  at  which  such  reviews  are  conducted.  Even  if 
reliability  design  reviews  are  to  be  conducted  separate  from  engineering 
reviews,  it  is  essential  that  close  coordination  be  maintained.  Such  coor¬ 
dination  is  necessary  primarily  because  engineering  reviews  such  as  Sys¬ 
tems  Design  Review  (SDR),  Preliminary  Design  Review  (FDR)  Critical 
Design  Review  (CDR),  and  First  Article  Configuration  Inspections  (^ACI), 
as  detailed  in  various  Air  Fores  Documents  such  as  AFSCM  37  5-5  and 
ESDP  375-10  are  co.nducteo  with  specific  objectives  land  the  results  of 
these  reviews  are  used  as  the  basis  for  decisions  such  as  those  concern¬ 
ing  continuation,  changes,  and  acceptability.  The  context  of  reliability 
reviews  conducted  separately  from  engineering  reviews  should  follow 
the  broad  outline  presented  m  the  documents  mentioned  above  with  one 
possible  exception.  Where  possible  the  acceptance  of  resultant  designs 
from  a  reliability  point  of  view  should  be  made  in  bulk  (that  is  at  large 
component,  functional,  or  equipment  levei)  rather  than  on  the  basis  of 
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individual  design  packages.  Evaluation  and  acceptance  in  bulk  provides 
a  much  better  opportunity  for  making  trade-offs  and  determining  whether 
reliability  on  some  packages  can  be  relaxed  (with  attendant  cost /schedule 
savings)  because  other  packages  within  a  functional  unit  have  been  de¬ 
signed  at  a  higher  than  anticipated  level  of  reliability. 

Detailed  direction  and  guidance  for  conducting  SDR's,  PDR's,  CDR's  and 
FACI's  are  presented  in  ESDP  375-10  (Instructions  For  Conducting  For¬ 
mal  Technical  Reviews,  Inspections,  and  Demonstrations);  AF5CM  375 
series  including  AFSC  375-  1,  Exhibits  I,  II,  and  XIV  and  AFSCM  375-5; 
and  AFR80-28(Engineering  Inspections). 

These  documents  present  review  requirements  pertinent  to  characteristics, 
in  general,  other  than  reliability.  However,  reliability  performance  and 
program  requirements  can  be  made  a  part  of  the  criteria  for  acceptability. 
Whether  integrated  with  engineering  reviews  or  conducted  separately  (but 
in  coordination  with  engineering  reviews)  it  should  be  understood  that  the 
design  review  should  be  planned  as  a  continuous  monitoring  of  a  design  to 
assure  tha.t  it  meets  the  expressed  and  implied  performance  requirements 
of  the  equipment  during  operational  use.  Such  reviews  provide  periodic 
appraisal  of  the  design  effort  to  determine  the  progress  being  made  in 
achieving  the  design  objectives  and  can  serve  to  systematically  bring  to 
bear  specialized  talent  on  specific  problem  areas.  In  addition,  overall 
evaluations  should  be  made  to  take  into  consideration  specific  design  and 
interface  problems  that  may  be  encountered  later  in  the  development  and 
production  cycle. 

GENERAL  DESIGN  REVIEW  GUIDELINES 

Techniques  for  designing  reliability  into  system /equipments  usually  are 
classified  into  three  categories: 

a.  Conservative  selection  and  application  of  piece  parts. 

b.  Incorporation  of  redundant  replacements  and/or  alternate  modes  of 
operation. 

c.  Minimization  of  environmental  stresses;  for  example,  electronic 
equipment  must  incorporate  means  for  adequate  heat  rejection  in 
order  to  provide  reliable  performance  at  thermal  equilibrium.  It 
cannot  be  overemphasized  that  high  reliability  can  be  achieved  only 
if  the  electronic,  thermal,  and  mechanical  designs  are  well  ex¬ 
ecuted.  The  thermal  design  is  fully  as  important  as  the  circuit 
design.  Ground-based  electronic  equipment  is  frequently  installed 
in  shelters  having  a  ventilating  or  air  conditioning  system  intended 
for  the  comfort  of  operating  personnel.  The  cooling  system  for  the 
electronic  equipment  must  be  made  compatible  with  such  a  system. 


Design  reviews  should  begin  with  the  conceptual  phase  that  considers  the 
broad  general  requirements  and  as  the  design  approaches  the  hardware 
stage,  narrows  down  to  detailed  meetings  of  reduced  scope  where  only- 
circuits,  equipments,  or  portions  of  equipments  are  considered,  and 
then  broadens  again  as  the  various  equipments  are  integrated  into  a  sys¬ 
tem. 

Design  changes  during  the  early  design  review  phases  generally  require 
very  little  engineering  effort  since  it  usually  involves  only  paper  changes 
of  a  part,  dimension,  or  value,  although  redesign  of  components  might  at 
times  be  mandatory.  Design  changes  occurring  during  subsequent  design 
reviews  involving  changes  to  drawings,  modifications,  or  replacement  of 
existing  hardware,  replacement  of  field  supplies,  revision  of  field  manuals, 
or  retraining  of  factory  and  field  personnel  for  example,  will  be  consider¬ 
ably  more  costly  (100-1000  times)  although  the  probability  of  such  changes 
will  be  less  than  during  the  first  phase.  As  it  pertains  to  reliability,  the 
periodic  review  of  design  at  key  points  in  the  development  program  facili¬ 
tates  detection  and  correction  of  actual  or  potential  design  problems  prior 
to  finalization  of  the  design. 

The  prime  purpose  of  a  formal  reliability  design  review  meeting  must  be 
to  insure  that  adequate  effort  in  reliability  is  being  made.  The  design  re¬ 
view  process  assures: 

a.  A  means  of  solving  interface  problems; 

b.  Confidence  "hat  experienced  personnel  are  involved  in  the  design  detail; 

c.  A  record  of  why  decisions  were  made; 

d.  A  knowledge  that  systems  will  tie  together  and  be  compatible; 

e.  A  total  picture  for  the  benefit  and  use  of  the  final  decision-maker  in 
making  trade-off  decisions;  and 

f.  A  greater  probability  of  a  fully  mature  design. 

A  design  review  plan  would  include  the  time-phased  events  representing 
the  appropriate  milestones  at  which  formal  system/equipment  reviews 
are  made  at  major  decisions  points.  The  number  of  critical  decision  points 
will  vary  according  to  the  type  of  development  program  underway.  The 
broad  categories  are  sometimes  listed  as: 

a.  Conceptual  Design 

b.  Prelimina  ry  Design 

c.  Preproduction  Design 

d.  Production  Design 


These  review  points  are  keyed  to  major  events  and  consequently  reflect 
the  name  of  that  event.  It  is  well  to  bear  in  mind  the  objectives  of  these 
reviews  and  schedule  the  event  accordingly.  The  main  requirements 
that  can  be  applied  to  any  program  will  be  covered  by  three  or  four  major 
review  points,  namely: 

a.  Conceptual  Design  Review, 

b.  System  Design  Review. 

c.  Preliminary  Design  Review. 

d.  Detailed  Design  Review. 

Regardless  of  the  names  assigned  the  design  reviews,  specific  milestones 
or  decision  points  must  be  identified  where  formal  reviews  will  be  con¬ 
ducted. 

MIL-R-27542,  paragraph  3.5.  10,  defines  the  requirement  for  contractor 
engineering  design  reviews  for  reliability.  This  paragraph  requires  the 
submission  of  a  schedule  of  planned  reviews  to  a  procuring  activity  and 
permits  the  attendance  of  procuring  activity  personnel  at  these  reviews. 

The  technical  ability  of  personnel  required  to  participate  in  these  re¬ 
views  will  vary  according  to  the  complexity  of  the  system. 

However,  conceptual  and  system  design  reviews  should  be  performed  by 
experienced,  senior  engineers.  Detailed  equipment  design  reviews  should 
be  performed  by  engineers  more  closely  associated  with  circuit  design, 
parts  application,  etc.  Again,  the  actual  number  of  personnel  participating 
in  formal  design  reviews  should  be  kept  to  a  minimum  commensurate  with 
the  specialists  required  for  the  problems  to  be  considered.  When  such 
specialists  as  metallurgists  or  comparable  authority  are  required,  they 
should  be  scheduled  to  join  the  group  at  a  specific  time  and  then  be  dis¬ 
missed  as  soon  as  possible. 

It  is  also  important  to  be  alert  to  contractor  methods  of  budgeting  for 
scheduled  design  reviews  to  assure  that  costs  are  not  compounded  by 
each  department  participating  in  reviews.  Design  review  costs  will  nor¬ 
mally  be  proportional  to  the  complexity  of  the  equipment  which  dictates 
the  number  of  review's  required  as  well  as  the  number  of  personnel  attend¬ 
ing.  A  rule-of-thumb  figure  sometimes  applied  is  that  design  reviews  re¬ 
quire  5%  of  the  overall  design- man- hour s .  The  effort  specifically  oriented 
toward  reliability  is  only  a  portion  of  this. 

Design  review  milestones  should  be  identified  early  in  the  program  and 
should  normally  be  coincident  with  the  main  development  phases.  The 
review  points  identified  should  be  firmed  up  approximately  30  days  in 
advance  of  a  formal  design  review  and  data  packages  should  be  distributed 
to  all  attendees  along  with  formal  notification  ten  days  prior  to  actual  date 
a  review  is  to  be  held.  The  specification  MIL-R-27542  requires  the 
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contractor  to  notify  the  AF  procuring  agency  t  n  days  in  advance  .< 
meeting  so  that  they  may  participate  if  they  so  ciesir<-.  In  an>  -<  ,  the 
minutes,  agenda,  actions,  and  documentation  should  if  available  r<  r 
review  when  requested  by  rhe  procuring  agency. 

Specific  information  that  must  be  reviev/ed  and  monitored  durn  ■  a  eon- 
tractor  design  review  program  includes: 

a.  Personnel  (their  experience  levels)  assigned  to  the  prog ’mm. 

b.  Organizational  assignments,  modus  operandi,  authority  delegated 
to  the  Design  Review  Board. 

c.  Design  handbooks  and  check  lists  prepared  for  design  engineering  use. 

d.  Design  review  plan--milestone  identification,  etc. 

e.  Data,  packages  developed  for  design  review  use.  These  packages 
should  include,  as  required,  worst  case  studies,  circuit  analysis, 
parts  application  data,  drawings,  etc.  The  completeness  of  packages 
is  very  important  for  individual  use  in  preparation  for  design  re¬ 
views  . 

f.  Recorded  actions  by  a  Board  including  rejected  recommendations  with 
reasons  for  rejection. 

g.  Approved  design  changes  and  their  documentation. 

h.  Records  indicating  problem  areas  not  resolved  at  the  meeting,  as  sign- 
ments  for  resolution,  specific  problems  to  be  studied,  target  dates 
for  completion,  and  methods  of  follow-up  to  assure  completed  actions. 

i.  Final  approval  of  design  by  respective  specialists  by  affixing  signature 
on  Board  minutes. 

The  conceptual  design  review  is  the  most  important  design  review  to  be 
accomplished.  Important  decisions  are  made  at  this  time  that  preclude 
or  freeze  subsequent  designs.  It  is  therefore  logical  that  this  review 
should  be  attended  by  the  largest  group  of  knowledgeable  engineers.  This 
Design  Review  Board  must  consider  the  feasibility  of  design;  the  techniques 
to  be  employed  in  achieving  performance  requirements;  the  interface  prob¬ 
lems  which  involve  system  maintenance,  and  design  concepts;  find  specific 
design  requirements  that  might  conceivably  push  the  Sate-of-the-art.  Major 
design  characteristics  such  as  performance,  reliability,  maintainability, 
and  value  must  be  carefully  considered.  A  proposed  configuration  should  be 
reviewed  for  such  considerations  as  the  use  of  standard  circuits  of  proven 
reliability,  comparison  of  one  computer  manufacturer  with  another,  evaluation 


of  belt-drive  versus  direct- 'Wive,  the  need  for  red:indant  replacements, 
the  hardware  approach  to  be  followed  in  the  identification  and  localization 
of  system  failures,  methods  to  minimize  the  influence  of  limited  or  critical 
life  items  on  the  operational  capability  of  the  system/equipment,  etc.  With 
such  considerations  a  paper  study  may  be  accomplished  to  obtain  an  esti¬ 
mate  of  the  system's  '-.''liability,  maintainability,  or  other  figures  of  merit. 
This  study  is  then  available  as  a  valuable  tool  to  assist  the  Board  in  select¬ 
ing  the  ultimate  system  configuration.  Although  early  r  lews  cannot  be 
rigorous  in  design  detail,  the  early  design  decisions  are  extremely  impor¬ 
tant  for  these  decisions  commit  the  program  to  a  specific  design  approach  or 
strategy.  Imprope  >gic  or  design  approaches  should  be  ferreted  out  at 
this  point  whe  re  changes  involve  only  paper  changes  and  before  actual  equip¬ 
ments  begin  to  take  shape.  As  the  design  progresses,  subsequent  changes 
become  much  more  expensive  and  tedious  to  accomplish. 

The  material  or  data  that  should  be  available  for  use  by  the  Board  in  its 
preparation  and  deliberation  includes: 

a.  The  proposal; 

b.  The  .jpecific  Operational  Requirement  (SOR); 

c.  The  Statement  of  Work  and  associated  specifications; 

d.  The  analysis  of  system  requirements; 

e.  Basic  design  criteria  (block  or  logic  diagrams  and  flow  charts); 

f.  Reliability  and  maintainability  requirements; 

g.  Possible  trade-off  documentation;  and 

h.  System/equipment  schedules  with  milestones. 

An  important  outcome  of  conceptual  or  system  design  reviev/s  is  the 
ability  of  the  systems  contractor  to  quantify  reliability  (and  maintainability) 
requirements  ai.  the  subsystem  level  for  the  guidance  of  design  engineer¬ 
ing  personnel  and  for  insertion  into  subcontracted  equipment  specifications. 

Subsystems  ana  component  detail  design  reviews  are  concerned  with  de¬ 
termining  the  reliability  characteristics  of  subsystems  during  the  detail 
design  phase  of  program  development.  The  purposes  of  this  effort  arc 
to  determine  the  extent  to  which  the  various  designs  in  process  will  achieve 
the  requirements  set  forth  as  the  result  of  the  conceptual  or  system  de¬ 
sign  review  and  to  indicate  the  need  for  redistribution  of  system  reli¬ 
ability  requirements. 
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The  number  of  formal,  retailed  subsystem  design  reviews  scheduled  and 
the  optimum  time  for  reviews  will  vary  as  a  function  of  the  s/stem  com¬ 
plexity,  the  type  of  equipment  being  utilized,  the  caliber  of  cognizant  de¬ 
sign  engineers,  i-ic,  lowever,  formal  design  reviews  should  be  conducted 
prior  to  release  of  an  design  to  production.  At  major  review  points  every 
facet  of  the  design  considerations  should  '  carefully  gone  over.  A  design 
review  check  list  should  be  utilized  to  assure  consideration  of  all  important 
criteria.  A  check  list  may  neces  -.a  rily  be  tailored  to  fit  the  specific  re¬ 
quire  >>nts  >  .  system,  but  m  any  case  it  should  not  be  a  different  set  of 

criteria  from  Lhe  -  es  used  by  designers.  It  would  be  unreasonable  to  con¬ 
front  the  designer  ith  a  new  set  of  rules  at  the  time  of  review.  In  some 
instances,  depending  on  the  size  and  o  ,pe  of  the  program,  it  may  be  appro¬ 
priate  to  pioviuc  or  have  the  contractor  levelop  a  reliability  design  hand¬ 
book  to  refbu  t  the  specific  reliability  requirements  of  the  project  to  de¬ 
sk  o  i  eineer  Such  design  handbooks  should  be  reviewed  for  adequacy 
ot  nuent  an'1  o  ceptabil ity. 

Development  Engineerii  1  personnel  of  the  Contract  Management  Regions 
should  be  fully  utilized  to  provide  continuous  surveillance  of  the  design 
review  effort.  This  source  of  engineering  talent  is  important  to  the  SPO 
effort  and  should  not  be  overlooked-  -  their  contribution  tan  be  of  great 
value  to  the  overall  effort. 

In  any  event,  design  reviews  should  not  be  staged  affairs  that  reflect  the 
results  of  previous  meetings,  but  should  indicate  a  thorough  preparation 
and  attention  to  detail  by  11  participants.  The  design  engineer  should  be 
prepared  to  defend  all  de»  isions  reached  by  him  by  presenting  required 
studies  (including  breadboard  test  data,  if  available),  mathematical  models, 
and  engineering  calculations.  Me  should  be  prepared  to  defend  the  selection 
of  a  resistor,  for  example;  not  by  merely  stating  that  it  is  reliable,  but  by 
saying  this  resistor  was  chosen  because  it  is  a  standard  item  with  the  low¬ 
est  possible  cost  to  perform  the  required  function;  it  is  derated  to  25%  of 
its  normal  rating  for  the  following  reasons...;  it  is  considered  as  reli¬ 
able  as  any  item  available  based  on  the  present  state-of-the-art  and  is  ex¬ 
pected  to  give  a  long  trouble-free  life  or  MT  BF  of  X  hours. 

Engineers  attending  reviews  should  be  thorough  in  their  pro- review  analysis 
of  what  they  consider  as  potential  problem  areas  and  should  be  prepared  to 
indicate  in  detail  the  effects  of  their  recommendation,  It  is  also  important 
that  Board  members  notify  the  designer  of  areas  of  disagreement  in  suffi¬ 
cient  time  before  the  formal  meeting  to  allow  him  to  assemble  reference 
material  to  support  his  decisions,  thus  allowing  the  Revio.v'  Boa  i  d  to  consider 
thoroughly  both  sides  of  the  question.  The  complete  analysis  and  pre¬ 
sentation  cf  facts  rather  than  theories  enables  sound  decisions  to  be  reached 
in  the  shortest  period  of  time.  Examples  of  the  types  of  data  necessary  to 
facilitate  detailed  reviews  include: 
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System  rel iability  predictions . 


b.  Detailed  subsystem,  circuit  reliability  predictions. 

c.  Component  parts  lists  with  appropriate  test  information. 

d.  Parts  derating  and  application  data. 

e.  Parts  failure  rate  data  (or  sources). 

f.  Stress  analysis  results. 

g.  Failure  effects  analysis. 

h.  Statistical  analysis  of  circuit  (or  assembly)  performance  as  a 
function  of  parts  variability.  Error  and  tolerance  studies. 

i.  Reliability  aspect  of  redundant  parts,  assemblies,  subsystems, 
modes  of  operation  with  attention  to  switching  problems. 

j.  Consideration  of  potential  reliability  growth. 

k.  Documented  reliability  growth  plans. 

l.  Analyses  of  known  trouble  areas,  with  plans  for  corrective  action. 

m.  Technical  data,  including  equipment  physical  construction  and  profiles, 
block  diagrams,  schematics,  signal  flow  charts,  equipment  operating 
theory,  maintenance  philosophy,  operating  procedures  and  maintenance 
instructions. 

While  the  above  has  been  oriented  toward  formal  Design  Review  Board 
actions,  there  should  also  be  a  general  look  at  the  interaction  of  a  con¬ 
tractor's  reliability  and  design  organizations  during  program  monitoring. 
The  influence  of  the  reliability  organization  on  the  design  process  must 
not  be  felt  only  at  formal  Board  meetings.  A  continuous  interaction  on 
questions  of  design  strategy  should  take  place  between  these  organizations 
throughout  the  design  process. 

It  should  be  realized  that  reliability  as  well  as  other  characteristics  are 
affected  by  every  decision  made.  This  includes  the  choice  and  use  of  cir¬ 
cuits  and  component  parts,  their  arrangement,  the  environment  in  which 
the  equipment  will  be  used,  and  the  operation  of  t.he  equipment  under  field 
conditions.  It  should  be  emphasized  that  reliability  is  not  the  sole  responsi¬ 
bility  of  a  designer,  nor  should  the  design  review  be  oriented  in  a  direction 
such  that  designing  becomes  a  responsibility  of  the  design,  review  team.  Be¬ 
cause  of  increased  complexity  of  the  equipment  being  designed,  concurrency 


concepts,  and  new  devices  and  techniques  being  employed,  it  is  impossible 
for  a  design  engineer  to  maintain  excellence  in  every  technical  discipline 
affecting  the  design  proces  s .  The  reliability  design  review,  if  properly 
performed,  provides  one  of  the  most  powerful  and  effective  tools  available 
to  assure  reliability  program  effectiveness  through  monitoring  reliability 
as  a  design  characteristic  early  in  design  or  at  the  optimum  stage  of  de¬ 
velopment. 

The  document  Handbook  for  Reliability  and  Maintainability  Monitors, 

A.D611  577,  from  Defense  Documentation  Center,  which  has  formed  the 
basis  for  much  of  the  material  presented  abovq  contains  more  detailed  guide¬ 
lines  and  checklists  which  can  be  used  as  a  basis  for  scheduling  and  con¬ 
ducting  reliability  design  revie  .vs. 

6.  RELIABILITY  TESTING 

Reliability  testing, or  the  evaluation  of  test  data  from  a  reliability  point 
of  view,  provides  the  first  tangible  'hard1  results  concerning  the  reli¬ 
ability  of  the  design.  The  result  of  conducting  a  reliability  analysis  based 
on  test  data  is  thus  very  critical  since  it  serves  as  the  basis  for  many 
decisions  such  as  those  concerning  design  adequacy,  assurances  that  re¬ 
quired  reliability  under  field  conditions  will  be  met,  and  the  none  for 
desig-  changes.  Therefore,  the  use  of  test  data  for  reliability  analysis 
should  be  very  carefully  planned  and  evaluated  before  it  is  accepted  as 
a  means  of  evaluation  even  though  this  approach  probably  provides  the 
most  effective  and  positive  means  for  assuring  reliability  program 
effectivenes  s . 

Details  of  reliability  test  and  demonstration  from  an  acc ept- reject  anti 
test  time  point  of  view  are  presented  in  Chapter  10,  Reliability  Measure¬ 
ment.  However,  reliability  tests  or  test  data  which  can  be  used  to  evaluate 
reliability  many  times  can  be  applied  long  before  reliability  demonstrations, 
even  those  of  a  preliminary  nature,  are  required.  The  availability  of  such 
data  and  its  potential  usefulness  is  an  important  note  which  should  not  be 
overlooked  by  the  SPO  or  the  contractor  as  a  means  for  monitoring  reli¬ 
ability  progress.  Use  ol  such  data  can  be  used  by  the  SPO  as  a  cri¬ 
terion  for  evaluating  contractor  program  effectiveness  based  on  whether 
plans  have  been  made  for  acquiring  such  data  and/or  such  data  are  being 
used  as  it  becomes  available. 

There  are  various  sources  of  useful  data  depending  on  the  type,  size,  and 
scope  of  the  program  and  on  the  point  in  the  life  cycle  at  which  a  program 
may  be.  For  instance,  in  a  program  involving  the  purchase  of  off-the- 
shelf  hardware,  data  from  performance  tests  and  in-service  use  may  be 
readily  available.  Even  in  initial  stages  of  a  development  program  there 
is  usually  data  available  on  portions  of  the  design  under  operational  or 
performance/reliability  oriented  test  conditions.  Such  data  is  usually 
available  since  very  few  development  programs  involve  hardware  in  which 
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every  element  of  the  design  is  completely  new.  Even  on  those  por¬ 
tions  of  the  design  on  which  such  information  is  not  available  there  is 
usually  some  development  test  or  breadboard  test  data  which  has  the 
potential  for  being  used  to  evaluate  reliability. 

A  key  point  related  to  using  reliability  testing,  reliability  demonstration 
testing,  or  test  data  to  evaluate  reliability  is  proper  planning.  This 
planning,  in  general,  should  be  directed  toward  establishing  (1)  the 
means  for  recording  or  acquiring  all  relevant  data  including  time,  a 
description  of  test  conditions,  and  conditions  under  which  failures  occurred 
and  were  recorded,  and  (2)  a  definition  of  system,  equipment  or  com¬ 
ponent  failure,  a  id  (3)  the  amount  of  data  required  to  establish  meaningful 
and  valid  conclusions.  The  following  is  a  brief  listing  of  the  type  infor¬ 
mation  which  is  pertinent  to  a  reliability  analyses: 

1.  Identification  either  of  equipment,  system,  or  unit  under  test, 

2.  Test  conditions  and  environments  (bench  test,  room  ambient  ,  RFI, 
vibration,  etc .  ). 

3.  Elapsed  time  irdicator  readings  (standby  and  operate). 

4.  Date. 

5.  T est  resul ts. 

6.  Failure  symptoms. 

7.  Elapsed  time  indicator  readings  at  time  of  failure. 

8.  Corrective  maintenance  action  to  restore  operation  or  a  description 
of  the  cause  of  failure  identified  to  the  smallest  replaceable  item. 

Data,  particularly  that  which  is  acquired  outside  of  regular  or  scheduled 
reliability  tests  or  demonstration,  should  not  be  used  indiscriminately 
since  this  can  possibly  result  in  drawing  misleading  conclusions. 

In  the  plan  to  use  reliability  tests  or  other  test  data  as  a  basis  for  evaluating/ 
monitoring  reliability,  considerable  thought  should  be  given  to  using 
scheduled  engineering  tests  that  are  required  at  various  key  points  in 
a  program  life  cycle.  Tests  such  as  Category  I  Tests,  Category  II  Tests 
and  Category  III  Testa  often  can  be  monitored  to  produce  usefu1  reli¬ 
ability  oriented  data.  The  use  of  these  tests  as  a  means  for  acquiring 
useful  reliability  data  can  have  two  advantages.  One  advantage  is  that 
daca  can  be  provided  which  reflects  specific  configurations  and  inter¬ 
faces  which  are.  an  integral  part  of  the  system  so  that  reliability  can 
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be  evaluated  along  with  and  as  early  as  other  required  engineering /per¬ 
formance  characteristics.  Those  changes  which  may  be  required  can 
be  evaluated  with  respect  to  the  total  requirement,  and  they  may  be 
implemented  more  easily  and  less  expensively  when  identified  in  early 
stages.  The  second  advantage  is  that  data  oriented  reliability  evalu¬ 
ations  can  be  achieved  at  a  cost  which  is  usuallv  only  a  fraction  of  that 
required  to  conduct  separate  reliability  tests.  In  addition  to  using  these 
test'--  as  an  opportunity  to  gather  data  for  evaluating  reliability,  in  many 
casts  they  can  probably  be  used  to  simultaneously  fulfill  the  require¬ 
ment  for  a  formal  reliability  demonstration.  Some  approved  reliability 
demonstration  techniques  are  presented  in  ESDP  HO  -  5 ,  Verification  of 
Quantitative  Reliability  Requirements  (Decision  Criteria)  IS  November 
1963. 

7.  TYPE  OF  TESTS 

As  mentioned  above  there  are  requirements  for  conducting  various  types 
of  tests  many  of  which  are  primarily  engineering  orien^e  !.  Rroad  guid¬ 
ance  is  provided  in  AFR  80-14,  with  some  amplification  in  ESDP  375-2, 

A  Typical  Test  Plan  For  Electronic  Systems.  However,  implementation 
and  the  details  of  conducting  tests  and  recording  data  arc  problems  which 
must  be-  resolved  by  the  SPO.  In  addition,  AFSCM  j7';-3,  Chapter  6  and 
AFSCM  375-4,  Chapter  3  of  Part  4  provides  information  on  various  types 
of  tests  and  their  general  objectives.  Further  information  on  establish¬ 
ing  test  time  requirements  and  risks  is  presented  in  ESDP  80-5. 

In  general  there  are  two  categories  of  tests  which  can  be  used  to  provide 
information  for  supporting  reliability  evaluations.  These  are  the  measure¬ 
ments  tests  (i.  e.  ,  tests  designed  to  measure  reliability),  and  evaluation 
tests  (i.  e.  ,  tests  which  generally  result  in  a  regression  analysis  designed 
to  evaluate  relationships  between  environments  or  stresses  and  parameters 
which  Influence  the  reliability  of  an  item).  Froperly  used,  both 
categories  of  tests  can  be  used  to  provide  information  for  monitoring  reli¬ 
ability  progress  or  for  identifying  the  potential  areas  where  greater  con¬ 
centration  is  required  to  achieve  reliability  cbjccti\es.  However  it  should 
be  pointed  out  that  the  approach  to  planning,  analysis,  and  use  of  results 
depends,  in  a  large  measure,  on  the  category  of  test  being  conducted. 

Since  tesl  data  can  be  extremely  valuable-  in  monitoring,  i1  is  important 
to  be  able  to  identify  the  types  of  tests  that  are  often  applied.  These  tests 
(listed  below)  can  frequently  be  used  as  sources  of  reliability  oriented  in¬ 
formation  provided,  of  course,  that  planning  and  preparing  has  been  such 
that  the  appropriate  reliability  information  will  be  recorded  along  with 
information  normally  obtained  from  these  tests. 
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1.  Qualification  Test.  This  test  simulates  defined  environmental  con¬ 
ditions  with  a  predetermined  safety  tractor.  The  results  of  this  test 
indicate  whether  a  given  design  can  perform  its  function  within  the 
simulated  environment  of  a  system;  tests  at  this  (in.  are  usu.illy 
not  made  using  production  tooling  and  processes. 

1.  Preproduction  Test.  This  is  a  test  of  design  qualified  hardware  that  is 
produced  using  production  tooling  and  processes  which  will  be  used  to 
produce  the  operational  hardware.  No  production,  hardware  should  be 
accepted  prior  to  satisfactory  completion  of  this  test.  Test  objectives 
include  the  gaining  of  confidence  that  production  hardware  is  going  to 
work;  it  will  be  reliable;  it  can  be  maintained  and  supported  by  the 
Air  Force;  and  is  not  over  designed. 

3.  Lot  Acceptanc  :  Test.  This  test  is  based  on  a  sampling  procedure  to 
assure  that  the  product  retains  its  quality.  A  specified  number  of  items 
from  each  lot  or  group  are  withdrawn,  at  random,  ami  tested  to  establish 
that  the  functions,  tolerances,  and  material"  have  not  degraded.  No 
acceptance  or  installation  should  be  permitted  until  this  test  for  the  lot 
has  been  successfully  completed. 

4.  Individual  Acceptance  Test.  This  is  based  on  a  test  of  predetermined 
critical  items  to  verify  their  operational  characteristics  prior  to  as¬ 
sembly  into  subsystems.  Waivers  to  this  requirement  such  as  using  the 
end  item  acceptance  tests  is  not  recommended  as  a  production  ex¬ 
pediency.  This  test  should  be  capable  of  being  performed  on  the  same 
fixtures  used  for  preceding  type  tests. 

5.  Critical  Weakness  Reliability  Test.  This  test  determines  the  mode  of 
failure  when  equipment  is  exposed  to  environments  in  excess  of  the 
anticipated  environments.  By  this  testing,  <-.ltical  levels  can  be  de¬ 
termined  for  vibration,  temperature,  voltage,  cycles,  etc.  ,  which  will 
adversely  affect  the  component.  In  subsequent  tests  of  the  totaL  system 
in  which  a  stress  level  exceeds  the  expected  limits,  an  evaluation  of 
the  critical  weakness  tests  wiLl  provide  excellent  insight  as  to  what 
may  have  been  damaged  or  what  can  be  expected  to  fail. 
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It  should  be  pointed  out  that  the  assurance  of  reliability  program  effectiveness 
requires  a  continuous  monitoring  and  evaluation  based  on  various  data  developed 
either  through  design  analysis  or  through  test.  As  can  be  seen  from  above,  a 
considerable  amount  of  test  data  which  is  particularly  useful  as  a  means  of  eval¬ 
uating  reliability  can  often  be  made  available  in  early  stages  through  proper  plan¬ 
ning  and  utilization. 

EVALUATING  RELIABILITY  PROGRAM  PLANS  AND  ORGANIZATION 

One  of  the  first  steps  in  assuring  reliability  program  effectiveness  is  a  thorough 
and  in-depth  analysis  of  the  reliability  program  plan  being  offerred  by  a 
contractor  and  that  which  is  subsequently  implemented.  In  general  the  require¬ 
ments  for  providing  data  describing  the  reliability  program  plan  are  presented 
in  Data  Item  R-l  of  AFSCM/AFLCM  310-1.  As  delineated  in  the  Data  Item  R-l 
a  reliability  program  plan  should  include: 

1.  A  detailed  listing  of  specific  reliability  oriented  tasks,  analysis, 
and  reviews  including  any  plans  for  parts  improvement  effort  or 
for  new  (or  modified)  military  specifications  which  contain 
reliability  levels  being  used  by  the  contractor. 

2.  A  description  of  the  general  procedures  for  implementing  and 
controlling  these  tasks  together  with  schedules  where  appropriate. 

3.  The  means  and  schedule  to  be  used  for  reporting  accomplishment 
of  these  tasks . 

4.  A  description  of  hosy  required  quantitative  reliability  objectives 
will  be  met  during  development  and  manufacture. 

5.  A  description  of  the  organization  and  personnel  responsible  for 
managing  the  overall  reliability  program. 

6.  A  description  of  the  responsibilities  and  functions  of  this  organization 
and  the  authority  delegated  to  it. 

7.  A  description  of  the  relationship  between  line,  service,  staff  and 
policy  organizations. 

8.  A  description  of  the  failure  reporting  system  to  be  used  including 
flow  charts  for  analysis,  feedback  of  corrective  action. 

9.  A  list  of  all  equipments  on  which  failure  reports  will  be  initiated 
and  the  point  in  time  when  failure  reporting  will  commence  for 
each. 
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10.  A  description  ol'  the  plan  for  demonstrating  at  a  specified  time 
achieved  reliability  including  estimated  number  of  test  articles 
and  confidence.  This  plan  should  include  trade-off  curves  showing 
number  of  test  articles  or  cost  versus  confidence,  and  will  include 
testing  at  the  system,  major  element  levels,  such  as  a  flight  vehi¬ 
cle,  and  major  subsystem  or  component  levels  separately  and  in 
combination,  as  applicable. 

11.  A  description  of  the  means  to  be  used  for  ensuring  that  appropriate 
reliability  requirements  are  included  in  subcontracts  and  the  methods 
to  be  used  for  establishing  such  requirements. 

Evaluation  of  a  reliability  program  plan  and  organization  can  be  considered 
to  take  plac  e  within  two  frames  of  reference. 

1.  Evaluation  with  respect  to  what  is  proposed  and  how  it  is  described. 

2.  Evaluation  with  respect  to  what  is  implemented  both  with  respect 

to  what  has  been  proposed  as  well  as  with  respect  to  what  should  be 
implemented  in  order  to  achieve  required  reliability. 

With  respect  to  evaluation  of  proposed  program  plans,  as  well  as  evaluation  of 
implemented  effort,  it  should  be  remembered  that  the  mere  existence  of  a 
reliability  program  will  not  increase  the  reliability  of  an  equipment,  but  an 
effectively  monitored  program  will  not  permit  an  inadequate  design  to  proceed 
into  development,  test,  production,  and  use  without  specific  management  approval. 

It  is  this  effective  monitoring  that  will  permit  project  engineers  to  assess  feasibility 
of  achievement  and  progress  in  time  to  make  adjustments.  Therefore,  the  process 
ol  evaluation  and  monitoring  should  be  continuous  in  order  to  ensure  effectiveness. 


Guidance  in  conducting  evaluation  can  be  developed  to  considerable  detail. 
However,  it  is  more  important  for  the  SPO  or  project  engineer  to  be  aware 
of  the  functional  and  organizational  and  task  relationships  that  are  important 
and  germane  to  a  reliability  program  plan  and  implementation.  For  in  this 
way,  the  SPO  or  project  engineer  can  fit  requirements  to  the  particular  pro¬ 
ject  at  hand.  This  offers  the  chance  to  provide  much  more  in  the  way  of  flex¬ 
ibility  and  sensitivity  to  items  which  may  be  peculiarly  important  in  significance 
or  priority  to  a  particular  project  either  from  a  technical,  schedule,  or  cost 
point  of  view.  The  following  pages  present  information  related  to  conducting 
evaluations  of  reliability  program  plans  and  organizations. 

Evaluation  ol  program  plans  and  organizations  should  be  heavily  influenced 
by  requirements  as  imposed  by  procurement  documents.  The  general  elements 
of  a  reliability  program  as  required  by  Data  Item  R-l  mentioned  earlier  form 
the  framework  for  development  of  a  program  plan.  However,  in  making  an 
evaluation,  criteria  other  than  those  elements  mentioned  above  should  be  used. 
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The  following  presents  certain  of  these  criteria  which  can  be  used  depending 
on  the  nature  of  the  program  from  a  technical  and  cost  point  of  view.  Use  of 
these  items  in  a  checklist  format  can  provide  an  effective  way  of  identifying 
program  plan  weaknesses  and  a  means  for  comparing  alternate  plans  in  order 
to  judge  potential  effectiveness. 

Does  the  program  plan  provide  for  or  include  a  description  of: 

(a)  Close  liason  between  personnel  in  the  reliability  program. 

(b)  A  specific  approach  to  re-liability  prediction. 

(c)  A  critical  effects  analysis. 

(d)  A  delineation  of  costs  for  reliability  program  tasks. 

(e)  A  description  of  the  procedures  to  be  used  in  conducting  design 
reviews . 

(f)  Indoctrination,  training,  and/or  motivation  of  personnel  in  reliability. 

(g)  A  data  collection  and  correction  action  program. 

(h)  A  schedule  for  submitting  timely  reports  on  results  of  analyses, 
predictions,  and  review. 

(:.)  Reliability  organizational  relationships  and  responsibilities  clearly 
defined  and  including  relationships  between  reliability  groups  and 
other  groups  within  the  company  organization. 

(j)  A  definite  means  for  reporting  reliability  status  to  top  management 
and  subsequent  actions  to  be  taken  and  a  delineation  of  data  sources 
to  be  us  ed. 

(k)  Identifying  and  analyzing  potential  reliability  problems  in  areas  such 
as  procurement,  design,  manufacturing,  quality  control,  testing  and 
logistic  support. 

(l)  The  procedure  to  be  used  in  conducting  critical  analyses,  the  criteria 
to  be  used  in  identifying  critical  items  and  the  specific  action  to  be 
taken  and  the  controls  to  be  used  to  insure  that  appropriate  action  is 
taken. 

(m)  The  tasks  or  work  elements  to  be  accomplished. 

(n)  The  work  to  be  accomplished  under  each  task  (task  description). 


(o)  The  time -phasing  of  each  task. 

(p)  The  man-loading  assigned  for  the  accomplishment  of  each  task. 

(q)  Appropriate  program  plan  milestone  review  points. 

(r)  Design  review  system,  its  method  of  operation,  responsibilities, 
and  authority. 

(s)  Corrective  action  system,  including  data  collection  system  and 
proposed  computational  ground  rules. 

(t)  Change  order  control  system,  with  particular  attention  to  the  method 
by  which  the  reliability  organization  has  the  opportunity  to  review  all 
design  changes  for  quantitative  effects. 

(u)  The  position  of  the  reliability  operation  within  the  management  struc¬ 
ture,  the  organization  of  this  operation,  and  the  channels  of  communi¬ 
cation  between  this  organization  and  design  engineering,  quality  control, 
test  engineering,  and  components  engineering. 

(v)  Alternative  tasks  as  substitutes  for  those  which  may  not  be  acceptable. 

(w)  Activities  defined  in  terms  of  functions  and  accomplishments  relating 
to  the  proposed  equipment. 

(x)  Planned  assignment  of  responsibilities  for  reliability  program  ac- 
complis  hments . 

(y)  Internal  "independent"  reliability  assessments  scheduled  to  coincide 
with  design  progress. 

(z)  A  reliability  demonstration  test  program  and  which  equipments, 
assemblies,  or  components  will  be  tested,  and  to  what  extent. 

From  a  subcontracting  point  of  view,  does  the  plan  include  a  description 

of: 

(a)  The  means  that  will  be  used  to  ensure  that  quantitative  reliability 
requirements  will  be  included  in  subcontracted  equipment  specifica¬ 
tions  . 

(b)  The  means  that  will  be  used  to  ensure  that  each  subcontractor  has  a 
reliability  program  which  is  compatible  with  the  overall  program. 
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(c)  The  procedure  for  reviewing  subcontractor  predictions  and 
computations  for  accuracy  and  correctness  of  approach. 

(d)  The  procedure  that  will  be  used  to  furnish  subcontractors  with 
failure  data  resulting  from  tests. 

(e)  The  plan  for  requiring  subcontractor  progress  reports  and  for 
reviewing  subcontractor  test  plans  for  accuracy  and  correctness 
of  approach. 

(f)  The  means  for  ensuring  that  subcontractors  have,  and  are  pur¬ 
suing,  a  vigorous  corrective  action  effort  on  causes  of  unreliab¬ 
ility. 

In  evaluating  reliability  program  plans,  it  is  also  important  to  evaluate 
the  organizational  structure  around  which  the  plan  will  be  implemented. 

One  factor  to  be  stressed  is  the  importance  of  retaining  the  designer  in 
the  information  and  reliability  activities  information  loop.  The  designer 
is  the  key  to  achieving  reliability  improvement  and  when  made  aware  of 
problems  and  given  adequate  information,  he  can  favorably  influence  de¬ 
sign.  In  general,  the  reliability  program  plan  should  identify  the  position 
of  the  Reliability  Group  or  Section  within  the  overall  organizational  structure. 
Usually,  the  Reliability  Group  either  is  positioned  as  a  line  activity  under 
engineering  or  combined  with  Quality  Control  (and  perhaps  other  discip¬ 
lines)  to  form  a  Product  Assurance  Department. 

Most  large  companies  will  have  a  Reliability  (or  Reliability /Maintain¬ 
ability)  Staff  Officer  (perhaps,  Vice-President,  Reliability  and  Quality 
Control)  who  is  responsible  for  generating  policy  and  standard  operating 
procedures. 

The  main  concern  in  studying  an  organization  is  to  determine  whether  or 
not  the  proposed  organization  will  be  responsive  to  the  overall  program 
requirements,  sensitive  to  problem  areas,  and  able  to  contribute  to  the 
formulation  of  design  criteria  and  the  control  of  design  for  reliability. 

Its  ability  to  perform  in  accordance  with  the  above  is  also  a  function  of 
its  personnel  capability -mix.  Since  reliability  encompasses  a  wide  variety 
of  tasks,  ranging  from  complex  modeling  techniques  to  design  criteria,  a 
program  plan  should  include  information  on  the  quality  and  quantity  of  peo¬ 
ple  available  to  perform  the  proposed  program. 
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In  addition,  the  organizational  structure  should  be  such  that: 

(a)  responsibilities  for  corrective  action  have  been  specifically 
assigned; 

(b)  responsibilities  for  establishing  suspense  dates  for  completion  of 
the  required  action  are  clear; 

(c)  responsibility  for  follow-up  to  assure  that  actions  are  actually  taken 
is  delineated  and  that  authority  exists  for  ensuring  response, 

(d)  responsibility  for  the  assessment  of  the  quantitative  effect  on  reli¬ 
ability  of  actions  taken  is  vested  in  a  position  of  responsibility  to 
accept  or  reject  the  approach; 

(e)  there  is  point  within  the  organizational  structure  where  progress 
against  various  problem  areas  can  be  monitored. 

Information  kept  at.  this  point  should  include: 

1.  Definition  or  statement  of  a  problem. 

2.  Corrective  action  contemplated. 

3.  Action  agency  or  responsibility  for  problem  resolution. 

4.  Effect  of  problem  on  reliability. 

5.  Action  completion  date. 

9.  EVALUATION  OF  IMPLEMENTED  PLANS  AND  ORGANIZATIONS 

Evaluation  of  implemented  plans  and  organizations  depends  to  a  large 
extent  on  the  plan  and  organization  structure  that  has  been  proposed  and/or 
approved  by  the  SPO.  During  implementation,  effort  should  be  primarily 
directed  toward  making  evaluations  between  what  has  been  implemented 
versus  what  was  proposed.  Significant  changes  should  be  explained.  The 
true  test  of  the  effectiveness  of  the  program  and  organization,  of  course, 
is  the  impact  on  design  and  the  resolution  of  problem  areas.  However, 
assuring  reliability  program  effectiveness  also  requires  personal  first-hand 
monitoring.  This  monitoring  should  include  discussions  with  technical  and 
management  personnel,  examination  of  internal  communication  structure  and 
the  responsibility  chain,  an  examination  of  the  flow  of  information  for  pro¬ 
blem  recognition  to  resolution  and  acceptance,  and  a  first-hand  examination 
of  the  way  in  which  informal  as  well  as  formal  design  reviews  are  conducted, 
as  well  as  the  way  in  which  the  requirement  for  test  data  is  planned  and  sat¬ 
isfied  . 
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The  following  briefly  describes  those  evaluations  which  can  be  used  to 
examine  implemented  plans  and  organizational  structures.  This  same 
approach  can  be  used  with  respect  to  various  other  items  that  are  part 
of  the  overall  plan. 


Engineers'  predictions  or 
e  stimate  s 

versus 

Published  predictions 

Data  feedback  from  tests 

versus 

Test  log  and  test 
technicians'  obser¬ 
vation  s 

Engineers'  description  of  design 
reviews 

versus 

Program  plans 

Personnel  from  whom  designers 
obtain  reliability  assistance 

versus 

Organizational 

structure 

Actual  company  -  sponsored 
reliability  training  of  technical 
personnel 

versus 

Documented  company 
training  program 

Actual  availability  of  data  on 
standard  parts  from  past 
experience 

versus 

Stated  or  implied 
avail?  bility 

Designer's  knowledge  of  reliability 
requirements,  including  environ¬ 
ments  and  performance  limits 

versus 

actual  requirements 

Procurement  personnel's  con¬ 
siderations  in  vendor  selection 

versus 

Program  plan 

Part  counts  and  stress  analysis, 
from  working  drawings 

versus 

Those  presented  in 
prediction  report 

As  mentioned  earlier  assuring  reliability  program  effectiveness  is  a  continuous 
process  starting  with  the  development  of  sound  reliability  requirements  and 
continuing  through  participation  in  design  reviews  and  test  demonstrations. 

The  entire  process  involves  analysis,  evaluations,  ^nd  decisions  based  on 
qualitative  information  such  as  that  contained  in  descriptions  of  proposed  plans 
and  that  resulting  from  analysis  of  implemented  effort  and  organizational  struc¬ 
ture  as  well  as  on  quantitative  information  such  as  that  resulting  from  pre¬ 
dictions  and  tests.  Since  a  program  or  organizational  structure  can  only  pro¬ 
vide  the  framework  within  which  meaningful  activity  or  effort  can  be  carried 
out,  it  is  important  that  the  SPO  or  project  engineer  make  adequate  provision 
for  the  technical  support  required  to  conduct  the  monitoring  and  analysis 
needed  to  assure  reliability  program  effectiveness  through  the  delivery  of 
hardware  which  meets  reliability  requirements. 
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CHAPTER  7 

EIELD  DATA  COLLECTION  AND  REPORTING 

1.  INTRODUCTION 

The  previous  chapters  of  this  notebook  have  discussed  a  variety  of 
activities  related  to  the  management  of  a  reliability  program  through¬ 
out  the  Conceptual,  Definition,  Acquisition  and  Operation  phases  of 
system  development.  These  activities  include  test-  for  the  measure¬ 
ment  of  the  achieved  level  of  reliability  and,  therefore,  provide  data 
applicable  to  system  performance  evaluation  activities.  However, 
these  tests  are  generally  concludedi  before  the  system  is  accepted  by 
the  using  command,  at  which  time  the  system  is  placed  on  an  opera¬ 
tional  status. 

The  true  measure  of  equipment  and  system  reliability  can  only  be 
determined  after  extended  operation  in  the  actual  field  environment. 
This  "operational  reliability"  can,  and  often  does,  differ  significantly 
from  the  levels  predicted  or  demonstrated  during  system  develop¬ 
ment.  For  example,  reliability  demonstration  test  data  are  evaluated 
based  on  certain,  assumptions  concerning  modes  of  failure  and  dis¬ 
tributions  of  failure  data.  These  factors,  however,  are  based  on 
experience  gained  on  previous  systems.  A  new  system  .may  not 
follow  the  assumed  failure  pattern,  especially  when  this  new  system 
involves  significant  advances  in  the  state  of  the  art.  Also,  operational 
reliability  is  often  strongly  influenced  by  unpredictable  stresses  in  the 
operational  environment. 

In  view  of  unavoidable  areas  of  uncertainty  such  as  these,  the  need 
for  extended  monitoring  and  evaluation  of  equipment  performance  in 
its  field  environment  becomes  apparent.  This  need  has  prompted  the 
establishment  of  formal  programs  for  the  continuing  collection  and 
reporting  of  reliability  data  generated  during  actual  field  operation. 
These  field  data  collection  and  reporting  programs  are  typically 
directed  toward  the  analysis  and  evaluation  of  reliability  data  gener¬ 
ated  during  the  Operational  Phase  of  the  system  life  cycle,  but  are 
also  applicable  to  analysis  of  data  collected  during  certain  Acquisition 
Phase  activities,  such  as  during  Category  II  testing  when  field  opera¬ 
tional  conditions  are  simulated. 

Certain  important  characteristics  of  a  field  data  collection  and  re¬ 
porting  system  are  discussed  in  this  chapter.  This  discussion  is 
intended  to  review  the  more  important  considerations  in  the  manage¬ 
ment  of  a  field  data  collection  and  reporting  program.  More  detailed 
information  concerning  the  design  and  implementation  of  such  systems 
can  be  obtained  in  references  listed  in  Chapter  12. 
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CHARACTERISTICS  OF  A  FIELD  DATA  COLLECTION  AND 
REPORTING  SYSTEM 


Ideally,  the  most  effective  field  data  collection  and  reporting  system 
is  tailored  to  the  specific  requirements  of  the  system  under  considera¬ 
tion.  However,  because  of  the  large  number  and  wide  variety  of  Air 
Force  systems,  a  policy  of  providing  a  unique  data  collection  and  re¬ 
porting  procedure  for  each  system  would  be  impractical.  Therefore, 
the  Air  F'orce  has  instituted  a  general  purpose  field  data  collection 
program  to  accommodate  all  operational  systems.  This  program, 
which  is  implemented  under  AFM  66-1,  "Maintenance  Data  Manage¬ 
ment,  "  is  intended  to  satisfy  a  variety  of  maintenance  management 
and  logistics  data  requirements,  including  the  analysis  of  operational 
reliability  factors. 


The  application  of  the  AFM  66-1  Maintenance.  Data  Collection  System 
in  the  collection  and  reporting  of  field  reliability  data  is  discussed  at 
the  end  of  this  chapter.  However,  due  to  the  generality  of  the  AFM  66-1 
system,  certain  supplemental  data  may  be  necessary  before  special¬ 
ized  reliability  analyses  can  be^pe rformed.  In  view  of  this,  the  dis¬ 
cussion  will  be  preceded  by  a  brief  discussion  of  the  chara-  (‘.eristics 
of  a  reliability  data  collection  and  reporting  cycle,  and  some  im¬ 
portant  factors  to  be  considered  in  designing  a  reliability  data  collec¬ 
tion  and  reporting  system. 


2.  1  Reliability  Field  Data  Collection  and  Reporting  Cycle. 


The  basic  functional  elements  of  a  typical  reliability  field  data 
collection  and  reporting  cycle  are  illustrated  in  Figure  7-1. 
Field  failure  reports  are  the  primary  vehicle  for  the  collection 
of  data  concerning  the  operational  reliability  of  systems  and 
equipments.  These  reports  not  only  provide  a  complete  record 
of  operational  failures,  but  also  contain  data  concerning  oper¬ 
ating  time,  environmental  conditions,  symptoms,  and  other  data 
that  are  essential  to  the  subsequent  analyses . 


Fcaw  field  data  are  prepared  and  processed  as  necessary  to  pro¬ 
vide  reliability  information  as  required  by  various  using  groups. 
This  processing  is  often  performed  by  computer  to  permit  timely 
analysis  of  the  large  volume  of  data  obtained  during  the  operation 
of  complex  systems.  Further  data  analysis  and  investigation  is 
often  necessary  to  interpret  the  computer  reports  in  terms  of 
hardware  design  or  system  program  management  information 
appropriate  for  correction  of  weak  areas  in  system  performance. 


Information  developed  by  design  or  management  groups  are  applied 
in  modifying  production  or  system  management  procedures. 
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Revised  Procedures  and  Policies 


Figure  7-l .  Basic  Elements  of  a  Reliability  Field  Oata  Collection  and  Reporting  Cycle 


Finally,  the  appropriate  redesigned  equipment,  modification 
kits,  or  management  policy  revisions  are  returned  to  the  field 
to  alleviate  the  reliability  problem. 

The  effectiveness  of  the  reliability  improvement  action  is  eval¬ 
uated  by  continuing  the  field  data  collection  and  reporting  program. 
Thus,  a  reliability  field  data  collection  program  becomes  a  con¬ 
tinuing  process  of  problem  detection  and  evaluation,  and  design 
improvement  monitoring. 

2.  2  Reliability  Field  Data  Collection  and  Reporting  System  Design. 

Reliability  data  from  field  operation  are  an  essential  aspect  of 
reliability  improvement.  Data  collection  and  reporting  is  not 
an  objective  in  itself,  however,  but  rather  is  an  essential  link 
in  the  chain  of  events  leading  to  the  ultimate  objective  of  product 
improvement.  Therefore,  the  design  of  an  effective  reliability 
field  data  collection  and  reporting  system  requires  consideration 
of  a  variety  of  factors  throughout  the  data  collection  and  reporting 
cycle.  These  factors  are  discussed  briefly  below  in  the  general 
order  in  which  they  should  be  considered.  The  first  three  of 
these,  as  summarized  in  Table  VII- 1,  form  the  foundation  for 
any  data  collection  and  reporting  system. 

a.  Objectives  of  Data  Collection  and  Reporting  System.  Belore 
any  data  collection  and  reporting  system  can  be  designed,  it 
is  es sential  that  the  ultimate  objectives  of  the  system  be 
clearly  stated.  The  objectives  of  reliability  data  collection 
and  reporting  systems  can  be  classified  in  three  general  in¬ 
formation  categories:  (1)  equipment  failure  patterns  and 
cause  and  effect  relationships,  (2)  parts  usage  information, 
and  (3)  operational  effectiveness  evaluations. 

b.  Analytical  Techniques  Required.  The  objectives  of  a  data 
collection  and  reporting  system  effectively  dictace  the  type 
of  analytical  technique s  that  will  be  required.  For  example, 
equipment  failure  cause  and  effect  relationships  are  deter¬ 
mined  by  means  of  analyses  directed  toward  determining 
the  frequency,  significance  and  identification  of  equipment 
failure  and  thus  will  require  statistical  analyses  of  failure 
data  to  establish  failure  distributions  and  cause  and  effect 
correlations,  and  can  involve  analysis  of  the  effect  of  part 
failure  on  overall  equipment  operation. 

Parts  usage  analyses  also  involve  statistical  analysis  of 
part  failure  data,  but  are  concerned  with  logistics  aspects 
such  as  part  provisioning  requirements  rather  than  equip¬ 
ment  performance.  Operational  effectiveness  analyses, 
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o.i  the  other  hand,  involve  evaluating  the  effect  of  equipment 
outage  on  the  operational  mission  of  the  overall  system. 

c.  Data  Input  Requirements.  The  third  important  consideration 
in  the  design  of  a  reliability  field  data  reporting  and  collec¬ 
tion  system  is  that  of  determining  specific  types  of  data 
required  in  performing  the  analysis.  In  general,  the  data 
requirements  are  dictated  by  the  type  of  analysis  to  be  per¬ 
formed  and  the  objectives  of  the  analysis.  For  example, 
equipment  failure  pattern  analyses  involve  part  identification, 
cause-of-failure,  and  part  operating  time  data.  Part  usage 
analysis  are  not  normally  concerned  with  cause-of-failure 
information,  but  require  more  detailed  data  concerning 
operating  time,  storage  time,  and  reprovisioning  time. 
Operational  effectiveness  analyses  require  data  concerning 
equipment  failure  or  degradation  with  respect  to  overall 
system  performance  and,  therefore,  do  not  usually  require 
detailed  part  data. 

The  variety  of  data  that  might  be  provided  by  a  typical  re¬ 
liability  field  data  collection  system  is  illustrated  in 
Table  VII- 2.  This  list  is  not  intended  to  be  complete  since 
specific  analyses  could  require  other  types  of  data.  Also, 
some  of  the  data  listed  may  not  be  essential  in  all  cases. 

d.  Design  of  Data  Collection  Procedures.  Once  the  data  re¬ 
quirements  have  been  established  it  will  be  possible  to 
design  a  data  collection  procedure  which  will  permit  the 
appropriate  reliability  field  data  to  be  acquired,  recorded 
and  presented  in  a  manner  that  will  facilitate  subsequent 
analysis.  The  major  element  of  a  data  collection  system 
is  the  field  data  report  form  on  which  raw  field  data  are 
recorded.  In  fact,  the  field  data  report  form  is  the 
communications  link  between  the  field  operation  and  sub¬ 
sequent  analysis  activities.  Thus,  a  well  designed  report 
form  is  one  of  the  key  elements  of  an  effective  field  data, 
collection  and  reporting  system. 

The  design  of  a  field  data  report  form  is  a  specialized 
activity  involving  the  consideration  of  practical  factors  of 
system  operation  and  support,  as  well  as  the  more  theoretical 
aspects  of  data  analysis.  In  any  case,  however,  a  field  data 
report  form  should  meet  the  following  general  criteria: 
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Table  VII-2.  List  of  Items  to  be  Included  in  Failure  Report  Form 


j  (1)  Report  number 

!  (2)  Original  report  number 

I 

(3)  Reporting  contractor 

(4)  Worl<  center  or  depaument 

(5)  System  type,  model,  series 

i 

|  (G)  System  serial  number 

j  (7)  Equipment  type,  model  designation,  model  number 
i  (8)  Equipment  serial  number 

(9)  Failed  item  part  number 

(10)  Failed  item  serial  number 

(11)  Failed  item  name  (noun) 

(12)  Failed  item  manufacturer 

(13)  Failed  item  reference  designation  (application) 

(14)  Next  higher  assembly  part  number 

(15)  Next  higher  assembly  serial  number 
(10)  Next  higher  assembly  name  (noun) 

(17)  Next  higher  assembly  manufacturer 

(18)  Next  higher  assembly  reference  designation 

(19)  Replacement  part  number 

(20)  Replacement  serial  number 

(21)  Subsystem 

(22)  Date  of  failure  (day,  month,  year) 

(23)  Operational  usage  at  failure  or  removal  (total  time/cycles/miles  by  category,  e.g., 
standby  or  operation,  and  environment) 

(24)  Total  age  of  item 

(25)  Activity  during  which  failed  item  discovered,  e.g.,  calibration,  checkout,  countdown, 
launch,  preflight,  flight,  etc. 

(26)  Initial,  subsequent,  and  final  disposition  (condemned,  repaired,  found  serviceable,  etc.) 

(27)  Effect  of  failure  (mission  failure,  performance  degradation,  no  significant  effect) 

(28)  Type  of  failure  (primary  or  secondary) 

(29)  Cognizant  action  agency 

(30)  Analysis  required  (yes  or  rio) 

(31)  Narrative  description  of  trouble  (how  malfunctioned) 

(32)  Failure  analysis  report  number 

(33)  Disposition  approval  signatures 
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The  form  should  permit  recording  of  all  essential 
data  in  a  format  compatible  with  the  analysis  to  be 
performed.  However,  superfluous  data  should  not 
be  recorded. 


The  form  should  be  conducive  to  accurate  data  re¬ 
cording.  Confusing  format,  inadequate  instructions 
and  other  sources  of  human  error  should  be  minimized. 


The  data  should  be  presented  in  a  format  suitable  for 
direct  use  in  subsequent  data  processing.  This  includes 
appropriate  coding  for  computer  processing  when 
wa  r  ranted . 


Data  Processing.  Reliability  data  processing  may  be  accom¬ 
plished  either  manually  or  by  electronic  data  processing 
methods.  The  data  processing  procedures  used  are  usually 
dictated  by  economical  factors  and  by  the  volume  of  data  to 
be  handled.  In  general,  electronic  data  processing  is  appli¬ 
cable  in  processing  reliability  data  obtained  from  large-scale 
or  widely-used  systems, while  manual  processing  is  more 
appropriate  for  processing  data  from  small  scale  or  one- 
of-a-kind  systems.  For  example,  failure  data  from  opera¬ 
tional  tests,  such  as  a  Category  III  test,  can  usually  be 
processed  by  manual  means.  However,  the  same  system 
deployed  in  large  numbers' for  field  use  will  probably  generate 
sufficient  volume  of  data  to  justify  the  application  of  electronic 
data  processing  procedures. 


The  particular  data  processing  procedure  to  be  used  should  be 
established  at  the  time  the  analytical  techniques  and  data  in¬ 
put  requirements  are  established.  This  will  permit  the  de¬ 
sign  of  a  data  collection  system  that  is  compatible  with  the 
data  processing  system  as  well  as  with  the  input  data  re¬ 
quirements  . 


AIR  FORCE  MAINTENANCE  DATA  COLLECTION  SYSTEM 


The  Air  Force  has  instituted  a  maintenance-oriented  field  data  collec¬ 
tion  system  that  is  intended  to  provide  data  necessary  for  manage¬ 
ment  of  the  system  maintenance  resources.  This  system,  which  is 
described  in  Air  Force  Manual  AFM  66-1,  is  used  primarily  for  base- 
level  management  within  the  Chief  of  Maintenance  complex.  Thus, 
the  data  collected  are  primarily  related  to  maintenance  control  and 
maintenance  manpower  management.  However,  the  system  is  also 
designed  to  provide  data  to  the  Air  Force  Logistics  Command  (AFLC) 
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for  material  management  and  logistic  support  requirements,  'lhere- 
fore,  the  AFM  66-1  Maintenance  Data  Collection  System  provides  for 
the  collection  of  certain  types  of  data  that  are  useful  in  reliability 
analyses . 

Some  of  the  analyses  that  are  possible  using  data  obtained  by  the 
AFM  66-1  system  are: 

a.  Analysis  of  high  system  failure  rate  and  high  part  consumption. 

b.  Analysis  of  component  and  end  item  data  to  screen  out  parts  which 
are  exhibiting  a  wide  variation  in  failures  between  different  instal¬ 
lations  of  the  same  system. 

c.  Identification  of  unreliable  items  and  substantiation  of  product- 
improvement  action. 


The  primary  problem  in  utilizing  the  AFM  66-1  Maintenance  Data 
Collection  System  for  the  collection  of  data  for  reliability  analyses 
is  that  the  Maintenance  Data  Collection  Forms  (AFTO  Forms  210, 
211,  and  212)  do  not  directly  provide  quantitative  measurement  of 
time  to  failure,  elapsed  time,  and  other  data  that  would  provide  a 
direct  analysis  of  an  achieved  level  of  system  reliability.  However, 
this  problem  can  usually  be  alleviated  by  the  use  of  supplementary 
forms,  for  the  collection  of  the  additional  data.  With  the  use  of  com¬ 
puters,  data  from  various  sources  such  as  this  can  be  combined  to 
produce  meaningful  reliability  information  for  use  by  management 
and  design  groups. 
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CHAPTER  8 


RELIABILITY  ALLOCATION 


1.  INTRODUCTION 


Reliability  allocation  is  the  process  of  establishing  reliability  require¬ 
ments  or  goals  for  various  subdivisions  of  a  system.  The  objective 
of  performing  a  reliability  allocation  analysis  is  to  arrive  at  reason¬ 
able  goals  for  each  subdivision  to  assure  the  achievement  of  the  re¬ 
quired  level  of  overall  system  reliability. 


Reliability  allocation,  or  apportionment,  is  closely  related  to  reli¬ 
ability  prediction.  A  prediction  of  system  reliability  is  usually  ob¬ 
tained  by  determining  the  reliability  of  the  lowest  level  items  and 
proceeding  through  intermediate  levels  until  an  estimate  of  system 
reliability  is  obtained.  Reliability  allocation  begins  with  a  state¬ 
ment  of  the  overall  system  reliability  requirement,  and  apportions 
this  total  requirement  among  subsystems  and  ,ower  subdivisions  con¬ 
stituting  the  system. 


In  actual  application  there  is  consice rable  overlap  between  prediction 
and  allocation.  An  allocation,  usually  performed  early  in  a  develop¬ 
ment  program,  helps  to  establish  a.  design  approach  for  meeting  a 
system  reliability  objective.  As  the  design  progresses,  predictions 
are  performed  to  evaluate  the  degree  to  which  (Le  system  reliability 
objectives  are  being  met.  For  example,  during  the  definition  and 
early  acquisition  phase  of  the  system  life  cycle,  allocations  are  often 
performed  to  aid  in  the  development  of  alternate  design  approaches, 
while  predictions  are  performed  to  assess  the  impact  of  proposed  de¬ 
sign  changes  on  system  reliability. 


Some  of  the  advantages  of  reliability  allocation  are: 


a. 


Reliability  requirements  are  apportioned  among  the  various  parts 
and  units  of  the  system  before  system  design  becomes  committed 
to  a  particular  design  approach. 


b. 


Attention  can  be  focused  on  the  reliability  relationship  between 
various  subdivisions  cf  the  system  and  on  the  contribution  of 
each  to  overall  system  reliability,  early  in  the  design  stage 
when  design  changes  can  be  made  more  easily  and  economically. 


c . 


A  judicious  apportionment  based  on  pertinent  factors  will  result 
in  placing  realistic  reliability  requirements  among  subsystems 
and  lower  subdivisions  throughout  the  system. 
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d.  The  possible  need  for  specific  reliability  design  effort,  such  as 
the  application  of  redundancy,  can  be  established  during  the  con¬ 
ceptual  phase  and,  therefore,  can  be  considered  in  preparation 
of  the  developments  specifications. 

Although  allocations  are  performed  by  the  program  manager  to  set 
down  the  main  reliability  goals  and  guidelines,  apportionment  analysis 
can  also  be  an  activity  of  prospective  contractors  in  preparing  contract 
definition  proposals.  Familiarity  with  techniques  and  meaning  of  re¬ 
liability  allocation  can  place  program  managers  in  a  position  to  evaluate 
contractor  proposed  subsystem  and  equipment  reliability  objectives, 
the  methods  by  which  they  were  establish-ed  and  proposed  design  approaches 
to  meeting  reliability  requirements. 

Some  basic  allocation  techniques  are  discussed  in  this  chapter,  begin¬ 
ning  with  the  quantification  of  mission  requirements  in  terms  of  system 
and  subsystem  reliability,  and  continuing  through  the  allocation  to 
lower  subdivi  ions.  It  should  be  noted,  however,  that  these  techniques 
will  be  effective  only  to  the  extent  that  the  allocated  values  are  actually 
achievable. 

2.  MISSION  REQUIREMENTS  AND  SYSTEM  RELIABILITY 

As  previously  indicated,  reliability  allocation  is  an  important  input  to 
establishing  system  and  subsystem  design  approaches  for  meeting  re¬ 
liability  objectives.  These  objectives,  however,  are  not  always  de¬ 
fined  in  terms  conducive  to  direct  identification  of  quantitative  reliability 
requirements.  In  fact,  during  early  conceptual  and  definition  phases, 
when  allocations  are  most  beneficial,  system  reliability  requirements 
may  only  be  implied  in  the  operational  mission  description  or,  at  best, 
are  included  together  with  factors  such  as  maintainability  in  defining  an 
overall  availability  requirement.  Therefore,  mission-oriented  require¬ 
ments  must  be  expressed  quantitatively,  and  in  terms  of  parameters 
associated  with  reliability  measurement  before  an  effective  allocation 
can  be  performed. 

Interpreting  system  performance  requirements  in  terms  of  reliability 
is  one  of  the  functions  of  the  reliability  program  during  early  phases 
of  a  system  life  cycle  when  reliability  requirements  are  only  implied 
in  operational  mission  descriptions.  This  interpretation  must  be  per¬ 
formed  in  a  manner  that  will  permit  quantitative  allocation  of  system 
reliability  requir ements  among  various  subsystems.  This  requires 
prooer  interpretation  of  statements  defining  mission-oriented  per¬ 
formance  requirements, and  quantification  of  these  requirements  in 
relation  to  operational  time  parameters  in  such  a  way  that  compatible 
and  practical  reliability  requirements  for  the  system  can  be  established. 
Alternatively,  an  initial  trade-off  study  may  be  required  to  optimize  the 
balance  between  reliability  and  maintainability  in  achieving  a  specified 
level  of  availability. 
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2.  1  Typical  Performance  Requirement  Statement.? 

Performance  requirements  for  different  systems  vary  considerably 
depending  on  the  purpose  of  the  system,  and  the  expected  mission. 
Performance  requirements,  as  usually  specified  in  early  system 
requirements  documentation,  are  concerned  more  with  what  is  to 
be  done  than  with  how  it  is  to  be  accomplished.  For  example,  a 
ground  radar  system  might  be  required  to  achieve  a  probability 
of  mission  success  of  80%  of  detecting  a  10  square  meter  aircraft 
target  flying  at  an  altitude  of  50,  000  feet,  and  at  a  range  of  200 
miles.  These  requirements  are  quite  different  from  those  for  a 
surface  to  air  missile  system  which  might  require  a  90%  kill  prob¬ 
ability  (i.  e.  ,  mission  success)  on  a  target  having  a  speed  of  mach  2, 
and  at  an  altitude  of  100,  000  feet. 

Operational  mission  requirements  stated  in  terras  such  as  these 
do  not  directly  provide  system  reliability  requirements.  However, 
a  quantified  system  reliability  statement  can  usually  be  developed 
by  considering  such  performance  requirements  together  with  ad¬ 
ditional  information  defining  other  aspects  of  the  Tanned  mission. 

2.  2  Reliability-Oriented  Quantification  of  Mission  Requirements 

Performance-oriented  statements  of  mission  requirements,  such 
as  those  mentioned  as  examples  in  paragraph  2.  1,  usually  include 
information  from  which  some  form  of  system  reliability  require¬ 
ment  can  be  derived.  However,  such  statements  will  often  re¬ 
quire  a  certain  degree  of  interpretation  and  even  some  assumptions. 
For  example,  even  though  reliability  is  a  function  of  mission  time 
constraints,  the  intended  mission  may  r.ot  oe  defined  quantitatively 
ir.  relation  to  time.  In  fact,  many  systems  are  not  intended  to  per¬ 
form  uniquely  definable  missions,  but  rather  are  intended  to  meet 
a  variety  of  possible  mission  requirements.  In  these  cases,  mis¬ 
sion  time  factors  cannot  be  precisely  defined  and  certain  assumptic  ■ 
are  necessary. 

For  a  simplified  example  of  a  procedure  for  interpreting  perfor¬ 
mance  requirements  in  terms  of  reliability,  consider  the  perfor¬ 
mance  of  a  raddr  set  which  has  a  mission  success  requirement  of 
80%.  This  requirement  could  be  met  by  a  failure-free  system 
having  exactly  a  80%  probability  of  detecting  the  specified  target 
at  200  miles.  However,  any  real  system  will  have  some  probability 
of  failure.  Therefore,  the  design  of  such  a  system  must  be  such 
that  the  probability  of  detection  is  greater  than  80%  when  the  sys¬ 
tem  is  operational  to  compensate  for  the  chance  of  a  failure  during 
an  operational  requirement. 
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In  its  most  elementary  form,  the  problem  wculd  reduce  to  obtain¬ 
ing  a  probability  of  at  least  0.80  that  the  system:  ( 1 )  is  not  in  a 
failed  condition  at  the  time  the  target  approaches,  (2)  remains 
"up"  for  the  time  required  for  detection,  and  (3)  detects  the  tar¬ 
get.  If  the  system  is  designed  such  that  the  actual  probability  of 
detection  is  96%,  then  the  probability  of  being,  and  remaining  "up" 
must  exceed  0.  80/0.  96  or  83.  3%.  This  proba.bility  of  being  "up" 
and  remaining  in  the  "up"  state  is  a  function  of  the  availability  and 
reliability.  In  this  particular  case,  the  availability.  A,  can  be  de¬ 
fined  as  the  probability  of  being  "up"  at  any  given  time.  This  quantity 
is  a  function  of  restore  or  repair  rate  (maintainability)  as  well  as 
failure  rate  and,  therefore,  is  beyond  the  scope  of  this  chapter.  For 
the  purposes  of  this  example,  a  value  of  A  =  0.  90  will  be  assigned. 

The  required  reliability  can  now  be  established  using  the  relation¬ 
ship  A*  R  =  .  833.  Substituting  the  value  for  A  gives: 


_  0.  833 

"  0.  90 


0.  926 


Therefore,  to  meet  its  performance  requirements,  the  radar  sys¬ 
tem  should  have  a  reliability  of  at  least  .  926  for  the  time  required 
for  target  detection.  Further  definition  of  the  required  system  re¬ 
liability  would  require  more  complete  definition  of  mission  and  op¬ 
erational  time  constraints. 


2.  3  System  Reliability  Requirements 

System  reliability  requirements  are  usually  stated  with  reference 
to  a  required  probability  of  satisfactory  operation  over  a  stated 
interval  of  time.  Therefore,  allocations  at  the  system  level  are 
usually  performed  on  a  probabilistic  basis.  In  the  event  system 
MTBF  or  mean  life  requirements  are  specified,  conversion  to  a 
reliability  or  probability  of  survival  requirement  can  be  performed 
before  proceeding  with  an  illocation  analysis.  Such  conversions 
are  performed  using  techniques  involving  statistical  failure  distri¬ 
bution  functions  as  described  in  Chapter  10. 

At  times,  the  information  available  at  the  time  an  allocation  is  to 
be  performed  does  not  permit  a  direct  calculation  of  reliability 
requirements.  Such  a  case  might  exist  when  mission-oriented 
system  reliability  or  MTBF  requirements  cannot  be  established 
because  of  a  lack  of  mission  profile  data.  In  such  cases,  a  gross 
allocation  can  be  performed  based  on  techniques  such  as  the  re¬ 
liability  prediction  by  function  or  other  procedures  for  obtaining 
gross  reliability  estimates.  These  procedures  permit  an  estimate 
of  the  achievable  system  reliability  based  on  elementary  performance- 
oriented  information.  Several  procedures  for  performing  analyses 


| 


8-4 


to  obtain  gross  reliability  estimates  that  are  applicable  in  per¬ 
forming  allocations  are  discussed  in  Chapter  9.  (see  Table 
IX-Z). 

RELIABILITY  ALLOCATION  TO  SB BSYSTEMS 

Once  a  system  reliability  requirement  has  been  established,  it  will 
usually  be  necessary  to  apportion  the  overall  reliability  among  several 
subsystems.  Several  basic  techniques  are  available  for  allocating 
system  reliability  to  the  subsystems.  The  particular  technique  to  be 
applied  in  a  given  situation  would  depend  on  many  factors  such  as  the 
amount  and  type  of  data  available  and  the  overall  configuration  of  the 
system.  Some  of  the  technique:  available  are  described  below  in  the 
order  of  increasing  complexity.  It  should  be  noted,  however,  that 
the  allocation  schemes  presented  here  will  be  valid  only  to  the  extent 
that  the  final  allocated  figures  are  achievable  by  the  components  to 
which  they  are  assigned.  If  reliability  allocations  are  not  achievable, 
redundancy  may  be  required  to  meet  the  overall  system  objective. 
Reliability  allocations  in  redundant  systems  involves  complex  model¬ 
ing  procedures  and  iterative  analysis  techniques  that  are  performed 
to  trade-off  reliability  with  cost  and  weight  penalties.  Such  techniques 
are  beyond  the  scope  of  this  chapter  and  wilL  not  be  discussed  here. 
However,  some  of  the  procedures  mentioned  in  Chapter  9  for  degra¬ 
dation  analysis  prediction  techniques  are  similar  to  the  procedures 
that  would  be  used  in  these  complex  allocations. 

3.  1  Equally  Critical  Subsystems  in  Series  Configuration 

The  most  elementary  procedure  for  allocating  an  overall  system 
reliability  among  several  subsystems  assumes  that  all  subsystems 
are  effectively  connected  in  series  and  are  equally  critical  to  sys¬ 
tem  operation,  (i.  e.,  the  failure  of  any  one  subsystem  would  re¬ 
sult  in  system  failure).  Also,  it  is  assumed  that  all  subsystems 
are  equally  complex.  In  such  a  case,  the  system  reliability  is 
apportioned  equally  among  all  subsystems. 

In  a  system  with  equally  critical  and  complex  subsystems  connected 
in  series,  the  overall  system  reliability  (probability  of  survival)  is 
equal  to  the  product  of  !he  reliabilities  of  the  subsystems.  This  can 
be  expressed 


where: 


Rg  =  the  reliability  of  the  system 


R,,R_,,  .  .  .,  R  =  the  reliabilities  of  subsystems 
1  2.  n 

number  1,  2,  .  .  . ,  n  respectively. 

If  all  subsystems  are  equally  critical  and  complex,  then 
Rj  =  R =  .  .  .  R^.  In  this  case  the  system  reliability  is 

Rg  -  Rn,  where  R  is  the  reliability  of  any  one  of  the  n  sub¬ 
systems.  Thus,  each  subsystem  would  be  assigned  a  reliability  of: 

1  /n 


R  =  R 


Example: 


Assume  a  system)  consists  of  three  subsystems  of  equivalent  com¬ 
plexity  in  a  series  configuration.  If  the  required  system  reliability 
is  R^  =  0.  95,  the  reliability  allocation  to  each  subsystem  (i.  e.  ,  the 

reliability  goal  established  for  each  subsystem)  would  be: 

R  =  (0.  95)1  /3  =  0.  983 


3.  2  Non-Equivalent  Subsystems  in  Series  Configuration 


Very  often,  as  more  information  concerning  the  s\?.bsystems 
becomes  available  it  may  become  apparent  that  the  subsystems 
are  not  "equivalent"  and  that  each  subsystem  should  be  assigned 
a  different  weight  in  its  contribution  to  system  reliability.  One 
method  of  reliability  allocation  which  uses  a  weighting  to  account 
for  subsystem  complexity  is  described  below.  A  similar  technique 
can  be  devised  for  weighting  by  other  factors  such  as  criticality  or 
operational  priority. 


Relative  subsystem  weights  based  on  complexity  can  be  calculated 
using: 


W.  = 


1  C1  +  C2  +  ---CN 
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'f 

1 

1 

V 

l 

1 

1 

£ 

whe  re: 

th 

W.  is  the  weight  assigned  to  the  i  subsystem, 

I 

|  Ch  is  the  complexity  of  the  i  subsystem  as  measured  in  terms 

!  | 

of  a  particular  type  of  complexity,  _,uch  as  parts  count,  or  active 

a 

element  group  (AEG)  count. 

th 

The  reliability  allocation  to  the  i  subsystem  can  be  calculated 

using: 

w. 

l 

1 

R.  =  ;  R  i 

1  1.  SJ 

I 

where: 

1 

j  .  th 

R.  =  allocated  reliability  for  the  i  subsystem 

I 

|  '1 

R  =  required  overall  system  reliability 

r  v  s 

1 

W.  =  the  weight  assigned  to  the  i  subsystem 

L  ;  1 

I 

Example: 

1 

Assume  a  system  consists  of  three  subsystems  in  series,  and 

the  overall  system  reliability  requirement  is  0.  95.  Also  assume 
that  the  complexity  (AEG  count)  of  the  subsystems  are: 

1 

v 

Subsystem  1:  -  200 

1 

Subsystem  2:  =  300 

1 

Subsystem  3:  =  500 

^  The  weights  assigned  to  each  subsystem  for  the  purpose  of  re- 

9 

jj;  liability  apportionment  are: 

9 

200 

|  W1  "  200  +  300  +  500  ‘  °’200 

\  j  W2  "  200  +  300  +  500  ~  °‘  300 

W  -  500  _  0  500 

3  200  +  300  +  500 

oo 

1 

r 

' 

l. . 

Ft 

t 

| 

The  reliabilities  can  now  be  allocated  to  Subsystems  1,  2  and  3, 
respectively,  as  follows: 

0.  200 


Rl  = 

:«*•«] 

=  0. 

990 

_0. 300 

R2  = 

o.  95  j 

=  0. 

LD 

oo 

O 

0.  500 

p*  - 1 

= 

1  o.  95 

=  0. 

975 

These  values  can  be  checked  by  calculating  the  system  reliability 
as  the  product  of  the  subsystem  reliability  such  that 

(0.  9 90 ) ( 0 .  985)(0.  975)  =  0.  95 


3.  3  Consideration  of  Subsystem  Importance  and  Complexity 

A  study  by  the  Advisory  Group  on  the  Reliability  of  Electronic 
Equipment  (AGREE)  recommended  apportioning  system  reliability 
based  on  the  importance  of  the  subsystem  to  system  operation,  as 
well  as  the  relative  complexity  of  the  subsystems.  The  mean  time 
between  failures  is  apportioned  to  a  particui  r  subsystem  using  the 
expres  sion: 

k.  t. 

l  l 


m.  = 


where: 

th. 

rm  is  the  MTBF  of  the  i  subsystem 

th 

k.  is  the  probability  that  the  system  will  fail  if  the  i  subsystem 
fails 

til 

t.  is  the  operating  time  of  the  i  subsystem  during  the  specified 
mission 

til 

n.  is  the  total  number  of  modules  in  the  i  subsystem 

N  is  the  total  number  of  modules  in  the  system 

Rg  is  the  required  system  reliability 
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This  technique  is  applicable  to  series  systems  in  which  failure 
of  a  subsystem  will  cause  system  failure  with  probability,  k. 

The  quantity  n./N  is  the  relative  complexity  ol  the  i^  subsystem, 

and  can  be  calculated  in  the  same  manner  as  w^  in  paragraph  3.2. 
The  subsystem  operating  time,  t;  represents  the  time  the  i^h  sub¬ 
system  would  be  required  to  operate  in  completing  the  defined 
mission. 

Exa  mple: 

Consider  a  system  with  a  reliability  requirement  of  0.  9  for  a  ten 
hour  mission.  The  system  consists  of  three  subsystems  A,  B  and 
C.  Subsystem  A  consists  of  10  modules  and  operates  for  the  entire 
mission;  if  a  module  within  A  fails  the  system  fails  with  certainty. 
Subsystem  B  consists  of  20  modules  and  operates  for  five  hours 
during  the  mission;  if  a  module  within  B  fails  the  probability  of 
system  failure  is  0.  9.  Subsystem  C  consists  of  8  modules  and 
operates  for  the  entire  mission;  if  a  module  within  C  fails  the 
probability  of  system  failure  is  0.  5. 

Table  VTII-1  gives  the  value  uf  k,  n,  and  l  for  Subsystems  A,  B  and 
C  and  the  required  MTBF's,  based  on  the  reliability  requirement 
of  C.  9. 

Table  VIII- 1  .  Reliability  Apportionment  Based  on  the 

AGREE  Technique 


k 

n 

t 

m 

■HUB 

1 

10 

m 

362  hours 

B 

0.  9 

20 

H 

81  hours 

C 

0.  5 

_8 

N  =  38 

H 

226  hours 

As  a  check  on  the  apportionment  technique,  the  resulting  system 
reliability  may  be  calculated.  The  system  reliability  is  given  by: 

R  =  R(A)  R  (B)  R  (C) 
s 

The  reliability  of  an  individual  subsystem  is  determined  from: 

-t.  ,m. 

R(i)  =  1  -  k.  (l  -  e  17  1S\ 
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Substituting  for  specific  subsystems: 

10 

R(  A)  =  l-  l(l-e  V  0.  972 

_5 

R(B)  =  1  -  0.  9  (  1  -  e  81  )  =  0.  945 

10 

R(C)  =  1  -  0.  5  ^  1  -  e  '  j  =  0.  978 

The  system  reliability  is  then  equal  to  the  product  of  these  values, 
or  0.  90.  From  this  it  would  be  concluded  that  the  system  re¬ 
liability  requirement  could  probably  be  met  by  apportioning  re¬ 
liability  requirements  to  the  subsystems  as  indicated  above. 

4.  RELIABILITY  ALLOCATION  TO  EQUIPMENT  AND  LOWER  SUB¬ 
DIVISIONS 

The  techniques  mentioned  in  paragraph  3  for  allocating  system  re¬ 
liability  requirements  involves  considering  a  probability  of  system 
survival  requirement  and  establishing  compatible  subsystem  reliability 
requirements.  At  the  time  during  the  system  life  cycle  when  such 
allocations  are  performed,  the  data  necessary  for  more  definitive 
allocation  are  not  available.  However,  subsequent  allocations  to 
equipments  and  lower  levels  will  normally  be  performed  later  during 
the  development  cycle  when  more  is  known  about  the  system  and  sub¬ 
system  design.  For  example,  at  a  cert  un  stage  of  the  design,  pre¬ 
liminary  reliability  predictions  will  ha’  e  been  performed,  and  gross 
failure  rate  predictions  will  be  available.  At  this  stage,  it  is  often 
desirable  to  allocate  maximum  subsystem  failure  rate  limitations  to 
lower  levels  of  hardware  design  and  thereby  provide  detailed  design 
goals  to  be  stated  in  terms  that  are  free  of  mission  time  parameters. 
One  example  of  a  technique  for  allocation  of  failure  rates  rather  than 
reliability,  which  permits  direct  application  of  available  failure  rate 
data,  is  described  below. 

4.  1  Allocation  of  Failure  Rates  in  a  Series  Configuration 

One  useful  technique  in  allocation  of  failure  rates  to  lower  sub¬ 
divisions  considers  the  relative  complexity  of  the  various  sub¬ 
divisions.  This  technique  is  based  on  the  assumption  that  the 
failure  rate  of  a  series  system  with  N  subdivisions  is  the  sum 
of  the  subdivision  failure  rates.  If  the  maximum  allowable 
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system  failure  rate  is  given  by  \  ,  this  value  must  be 

max 

apportioned  so  that  the  subdivision  failure  rates  X.  satisfy 
the  following: 


X2  + 


XN  * 


max 


Let  the  estimated  complexity  of  the  subdivisions  be  given  by 
Cr  C2.  .  .  . ,  C^.  The  relative  element  weights  are  calculated 
using: 


w. 

J 


+  c2  +  . 


The  apportioned  failure  rates  are  then  determined  for  each  in¬ 
dividual  element  by: 


N 

Since  ,■  w. 
-  J 

j  =  l 


X. 

J 


w.  \ 
j  max 


(j  =  1,2 . N) 


1,  the  failure  rates  are  apportioned  so  that 


N 

A  V 

j  =  l 


X 

max 


Example: 


Consider  a  subsystem  consisting  of  three  equipments:  a  power 
supply,  a  receiver,  and  a  transmitter.  The  maximum  allowable 
subsystem  failure  rate  is  0.005  failures  per  hour  (an  MTBF  of 
200  hours). 


A  failure  in  either  one  of  the  three  equipments  will  cause  a  sub¬ 
system  failure.  The  complexity  of  each  equipment  has  been 
determined  as  follows: 


Power  Supply: 

Receiver: 

Transmitter: 


1  00  parts 
225  parts 
560  parts 


The  "complexity"  of  an  item  is  a  measure  of  the  number  of  elementary  parts 
making  up  the  item,  In  general,  complexity  is  determined  by  parts  count 
based  on  available  design  data.  If  necessary,  the  complexity  can  be  estimated 
based  on  known  complexity  of  similarly  constructed  items  performing  com¬ 
parable  functions. 
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The  weight  assigned  to  each  module  is  then: 


W 


(Power  Supply) 


_ 100 

100  +  255  +  560 


1 1 


W 


(Receiver) 


25  5 

100  +  255  +  560 


=  .  28 


W 


(Transmitter) 


560 

100  +  255  +  560 


.  61 


The  failure  rates  can  then  be  apportioned  as  follows: 

Power  Supply:  0.  11(0.  005)  =  0.  00055 
Receiver:  0.28(0.005)  =  0.00140 

Transmitter:  0.61(0.005)  ~  0.  00305 
System:  =  0.  00500 


5.  IMPLEMENTATION 


The  preceding  discussions  describe  some  of  the  techniques  commonly 
used  in  allocating  system  reliability  requirements  to  lower  subdivisions. 
The  primary  purpose  of  such  allocation  is  to  provide  a  guide  for  the 
development  of  reliability  design  goals,  and  for  providing  subsystem 
reliability  requirements  for  procurement  specifications.  For  ex¬ 
ample,  a  gross  system  reliability  requirement  can  be  allocated  to  the 
various  subsystems  in  a  manner  that  considers  practical  aspects  such 
as  complexity,  system  configuration  and  operational  priority.  Further 
allocation  of  subsystem  reliability  can  provide  compatible  failure  rate 
goals  applicable  to  the  design  of  lower  subdivisions. 

In  addition  to  providing  design  goals,  appropriately  developed  reliability 
allocations  can  provide  valuable  input  for  activities  such  as  reliability 
feasibility  analyses,  preliminary  reliability  modeling,  cost  effectiveness 
studies,  and  any  of  many  design  trade-off  analyses  performed  during 
system  development. 

The  techniques  mentioned  here  are  only  a  few  of  the  procedures  that 
come  under  consideration  when  performing  reliability  allocations  for 
complex  systems.  In  an  actual  case,  the  allocation  procedures  will 
be  selected  based  on  many  factors  such  as  system  function  and  mission 
requirements,  and  may  involve  complexities  such  as  the  consideration 
of  a  variety  of  missions,  and  missions  that  vary  with  time  (i.  e. the 
probability  of  needing  a  given  system  function  may  change  as  a  mission 
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progresses).  Therefore,  in  a  practical,  case  all  such  factors  must  be 
defined  before  an  effective  allocation  can  be  performed.  Appropriate 
techniques  are  usually  selected  as  a  part  of  the  overall  reliability  en¬ 
gineering  task. 

In  selecting  and  applying  the  allocation  techniques,  however,  it  should 
be  noted  that  the  resulting  allocations  will  be  valid  only  if  the  final 
allocated  values  are  achievable  by  the  components  to  which  they  are 
assigned.  If  these  allocated  levels  of  reliability  are  not  achievable, 
redundancy  will  be  required.  In  this  case,  a  much  more  complex 
scheme  must  be  designed  to  trade  off  reliability  with  cost  and  weight 
penalties  to  establish  an  optimized  system  configuration. 

Overall  system  reliability  requirements  are  usually  allocated  to  sub¬ 
systems  during  the  Conceptual  Phase  to  provide  reliability  require¬ 
ment  for  the  preliminary  technical  development  plan  (PTDP),  and  for 
the  contract  definition  phase  RFP.  Allocation  techniques  also  are  in¬ 
valuable  to  contractors  in  performing  trade-off  studies,  developing 
reliability  design  requirements  for  contract  end  items  (CEI),  and  per¬ 
forming  other  tasks  associated  with  proposal  development  during  con¬ 
tract  definition.  In  addition,  original  and  up-dated  allocations  provide 
the  basis  for  the  development  of  criteria  for  performing  the  technical 
evaluation  of  competing  contractors'  proposed  system  reliability  en¬ 
gineering  approaches. 
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CHAPTER  9 


RELIABILITY  PREDICTION 


INTRODUCTION 

Reliability  prediction  is  the  technology  of  estimating  the  level  of 
reliability  that  will  be  achieved  by  a  system  on  the  basis  of  func¬ 
tions  and  characteristics  of  the  system  design  and  the  expected  oper¬ 
ating  environment.  Reliability  predictions  provide  a  means  of 
quantitatively  assessing  system  reliability  before  hardware  models 
are  constructed  and  tested.  Thus,  they  are  an  important  aspect  of 
system  development  and  engineering,  and  have  important  applications 
throughout  the  conceptual,  definition  and  early  acquisition  phases 
of  the  system  life  cycle. 

The  several  reliability  prediction  methodologies  in  common  use  per¬ 
mit  reliability  analyses  to  be  performed  with  varying  degrees  of 
detail  and  provide  useful  evaluations  of  system  reliability,  even  while 
the  system  is  in  the  early  stage  of  development.  For  this  reason, 
reliability  predictions  are  ./aluable  tools  in  performing  a  variety  of 
essential  tasks  as  the  system  development  program  progresses. 

1.1  Reliability  Prediction  Applications 

System  development  tasks  involving  significant  prediction  activity 
can  bo  classified  in  seven  general  task  categories:  feasibility 
study,  allocation  study,  design  comparison,  proposal  evaluation, 
trade  study,  design  review,  and  design  analysis.  These  task 
categories  differ  in  principle  objective,  and  are  performed  at 
different  stages  of  the  development  program.  However,  the 
associated  reliability  predictions  are  performed  using  essentially 
the  same  basic  procedure  for  any  task  category;  the  major 
difference  in  procedure  being  dictated  by  the  particular  phase 
of  the  system  life  cycle  rather  than  by  the  objective  of  the  task. 
The  purpose  and  characteristics  of  the  various  task  categories, 
and  the  relationships  of  reliability  predictions  to  the  task  objec¬ 
tive  are  summarized  in  the  following  paragraphs.  These  tasks 
typically  are  the  responsibility  of  either  the  Air  Force  or  Con¬ 
tractor,  and  are  performed  during  different  phases  of  the  life 
cycle,  as  illustrated  in  Table  Df-l. 

a.  Feasibility  Studies.  Feasibility  studies  are  performed  as  a 
part  of  the  exploratory  development,  advanced  development, 
system  study,  and  system  engineering  activities  of  the  Con¬ 
ceptual  Phase.  In  general,  feasibility  studies  are  directed 
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toward  evaluating  the  feasibility  of  given  technological 
approaches  in  meeting  operational  objectives  within  the 
practical  constraints  of  time  and  cost.  However,  in  most 
systems  severe  operational  requirements  generate  addi¬ 
tional  constraints  in  the  area  of  system  effectiveness. 
Therefore,  an  evaluation  of  the  probable  level  of  system 
reliability  that  will  be  achieved  by  the  approach  in  question 
is  a  critical  factor  in  the  feasibility  analysis.  Such  evalu¬ 
ations  of  system  reliability  involves  reliability  predictions 
based  on  extremely  gross  system  performance  and  environ¬ 
mental  data. 

b.  Allocation  Study.  During  the  late  Conceptual  Phase,  and 
during  Contract  Definition,  a  series  of  progressive  studies 
are  performed  for  the  purpose  of  allocating  overall  system 
effectiveness  requirements  among  the  several  subsystems 
and  elements  of  the  system.  System  effectiveness,  which  is 
a  measure  of  the  system's  ability  to  perform  as  required, 
includes  system  reliability  as  a  major  parameter.  Ihere- 
fore,  a  significant  portion  of  the  allocation  studies  involve 
the  allocation  of  system  reliability.  During  such  studies, 
reliability  prediction  procedures  are  applied  as  the  primary 
analytical  tools  of  reliability  allocation.  (The  general  sub¬ 
ject  of  reliability  allocation  is  reviewed  in  Chapter  8.) 

c.  Design  Comparison.  During  the  Definition  Phase,  alternate 
designs  of  established  feasibility  are  compared  in  selecting 
the  particular  approach(s)  to  be  considered  for  further  de¬ 
velopment.  Such  comparisons  typically  are  performed  by 
the  contractors  during  the  contract  definition  studies,  and  by 
the  Air  Force  SPO  in  evaluating  the  results  of  the  various 
competing  contractors  efforts.  Reliability  is  a  major  param¬ 
eter  in  the  effectiveness  of  a  given  design  and  must  be  con¬ 
sidered  in  any  comparison  of  alternate  designs.  Thus,  the 
probable  level  of  system  reliability,  as  determined  by  means 
of  reliability  predictions,  become  an  important  aspect  in 
comparing  alternate  designs. 

d.  Proposal  Evaluation.  Evaluation  of  competing  proposals  for 
contract  definition  studies,  and  subsequent  evaluation  of 
Acquisition  Phase  proposals  resulting  from  the  contract  defi¬ 
nition  effort  involve  careful  analysis  of  many  factors  such  as 
cost,  contractor  capability  and  proposed  design  approach. 

In  general,  proposals  are  evaluated  in  terms  of  many  inter¬ 
related  factors.  One  of  these  factors  is  an  evaluation,  usually 
by  means  of  a  reliability  prediction,  to  verify  the  reliability 

9-3 


JaaiikttA*  nil,  jgj 


1. 


of  the  proposed  design.  Such  evaluation  requires  a  relia¬ 
bility  prediction  which  is  directed  toward  assuring  that  at 
least  a  minimum  acceptable  level  of  reliability  will  be 
realized.  This  is  not  normally  as  vigorous  as  a  prediction 
performed  for  the  purpose  of  estimating  the  actual  value  of 
reliability,  as  might  be  required  in  design  comparison  studies. 

e.  Trade; -Off  Studies.  Trade-off  studies  are  usually  an  integral 
part  of  the  development  and  design  activities  during  the  Defi¬ 
nition  and  Acquisition  Phases.  Trade-off  studies  can  involve 
any  of  many  sets  of  interrelated  design  factors.  However, 
in  most  cases,  physical  and  functional  design  characteristics 
are  traded  against  each  other  within  constraints  of  system 
operational  effectiveness ,  operational  requirements,  and 
physical  restrictions.  Thus,  reliability  predictions  as  the 
trade-off  areas  are  varied  are  important  aspects  of  any  trade¬ 
off  study  because  of  the  relationships  between  system  effec¬ 
tiveness  and  reliability. 

f.  Design  Review.  During  the  Acquistion  Phase,  official  design 
reviews  are  performed  at  appropriate  stages  of  system  develop¬ 
ment  to  assure  that  the  design  is  progressing  according  to  the 
baseline  requirements.  These  design  reviews  include  an  assess¬ 
ment  of  every  aspect  of  design,  including  the  achieved  level  of 
reliability.  Therefore,  a  reliability  prediction  performed  at  a 
level  of  detail  that  considers  elementary  details  of  hardware 
design  usually  forms  an  important  part  of  analysis  performed 

in  support  of  a  design  review. 

g.  Design  Analysis.  As  a  system  design  progresses  throughout 
the  development  stage  of  the  acquisition  phase  periodically 
updated  reliability  predictions  are  invaluable  to  the  contractor 
in  identifying  reliability  problems  that  may  be  generated, 
assessing  the  effect  of  design  change,  and  otherwise  measur¬ 
ing  the  progress  of  the  design  with  respect  to  the  achieved 
reliability.  In  addition,  specialized  design  analyses  involving 
reliability  predictions  are  performed  for  such  purposes  as 
supporting  maintenance,  logistics  and  test  program  planning 
studies. 

Reliability  Prediction  Techniques 

Several  reliability  prediction  techniques,  varying  in  level  of  com¬ 
plexity  and  detail  of  application,  are  available  to  the  reliability 
engineer.  In  general,  the  techniques  in  current  use  provide 
means  for  predicting  total  equipment  or  system  reliability  as  a 
function  of  its  defined  design  or  functional  characteristics.  Also, 
to  the  extent  possible  considering  the  maturity  of  available  data, 
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most  prediction  techniques  consider  the  statistical  distribution 
of  failures  to  permit  reliability  evaluation  in  a  quantitative 
manner. 

As  mentioned  in  paragraph  1.1,  reliability  predictions  have  a 
variety  of  applications,  and  musi  be  performed  at  various  times 
throughout  the  system  life  cycle  beginning  during  the  Conceptual 
Phase,  and  continuing  through  the  Definition  and  Acquisition 
Phases.  During  this  time  span,  data  describing  the  system  de¬ 
sign  evolves  from  a  qualitative  description  of  systems  functions 
to  detailed  specifications  and  drawings  suitable  for  hardware  pro¬ 
duction.  Therefore,  reliability  prediction  techniques  have  been 
developed  to  accommodate  the  different  reliability  study  and 
analysis  requirements  as  the  system  design  progresses.  These 
techniques  can  be  roughly  classified  in  six  categories,  depending 
on  the  type  of  data  or  information  considered  in  the  analysis. 

These  categories  are: 

a.  Similar  Equipment  Techniques.  The  equipment  under  considera¬ 
tion  is  compared  with  similar  equipments  of  known  reliability  in 
estimating  the  probable  level  of  achievable  reliability. 

b.  Similar  Complexity  Techniques.  The  reliability  of  a  new  design 
is  estimated  as  a  function  of  the  relative  complexity  of  the  sub¬ 
ject  item  with  respect  to  a  "typical"  item  of  similar  type. 

c.  Prediction  by  Function  Techniques.  Previously  demonstrated 
correlations  between  operational  function  and  reliability  are 
considered  in  obtaining  reliability  predictions  for  a  new  design. 

d.  Part  Count  Techniques.  Equipment  reliability  is  estimated  as 
a  function  of  the  number  of  parts,  in  each  of  several  part 
classes,  to  b.:  included  in  the  equipment. 

e.  Stress  Analysis  Techniques.  The  equipment  failure  rate  is 
determined  as  an  additional  function  of  all  individual  part 
failure  rates,  and  considering  part  type,  operational  stress 
level,  and  derating  characteristics  of  each  part. 

i.  Degradation  Analysis  Teclmiques.  Circuit  tolerances,  param¬ 
eter  drift  characteristics,  part  variation,  and  other  factors 
are  considered  together  with  stress  levels  in  predicting  the 
probability  of  circuit  malfunction  due  to  wear  out  or  other  types 
of  degradation. 

Reliability  prediction  techniques  in  each  of  these  categories  are 
described  in  subsequent  paragraphs  of  this  chapter  following  a 
brief  discussion  of  some  of  the  underlying  mathematical  princi¬ 
ples  of  reliability  prediction. 


MATHEMATICAL  FOUNDATION  OF  RELIABILITY  PREDICTION 


The  reliability  prediction  techniques  in  common  use  are  founded  on 
the  accepted  basic  definition  of  reliability,  i.e.,  the  probability  that 
a  system,  subsystem  or  equipment  will  perform  a  required  function 
under  specified  conditions  without  failure  for  a  specified  period  of 
time  (see  AFR  80-5).  This  probabilistic  definition  has  permitted 
the  application  of  basic  probability  theory  in  stating  the  fundamental 
concept  of  reliability  and,  from  this,  developing  the  mathematical 
foundation  of  reliability  prediction. 

2.  1  Probabilistic  Concept  of  Reliability. 

Consider  an  item  (system,  subsystem,  etc.  )  for  which  there  are 
n  possible  causes  of  failure,  and  where  C^  represents  the  i^v 
cause  of  failure  (i  =  1,  2,  ...  n).  If  the  probability  of  occurrence 
of  cause  Cq  is  P(Cq),  then  the  conditional  probability  of  the  occur¬ 
rence  of  a  failure  (F)  due  to  cause  Cb  is  P(F/Cq).  Applying  the 
rules  for  combining  statistically  dependent  events  (one  event 
must  have  occurred  before  the  other  can  occur)  the  probability 
of  the  cause  and  failure  occurring  is  P(Ci)P(F /Ci). 

This  represents  the  probability  of  a  failure  occurring.  However, 
reliability  is  a  measure  of  the  probability  of  performance  without 
failure.  Therefore,  the  basic  mathematical  definition  of  relia¬ 
bility  considers  that  a  given  cause  and  resulting  failure  will  either 
occur  or  not  occur  so  that  the  probability  of  the  item  failing  due 
to  cause  Cq  plus  the  probability  of  not  failing  due  to  cause  Cq  is 
equal  to  1.  By  definition,  the  reliability  of  the  item  with  refer¬ 
ence  to  cause  Cq  is  the  probability  of  not  failing  due  to  cause  Cq,  or 

R.  -  .1  -  P(C.)P(F/C.)  (1) 

i  ii 

Defining  the  reliability  of  the  overall  item  as  the  probability  that 
there  will  be  no  failure  due  to  any  of  the  n  possible  causes,  pro¬ 
vides  the  fundamental  concept  of  reliability  which  is  stated: 

R  -  tt  [1  -  P(C,)P(F/C.)1  .  (2) 

i-  1 

This  expression  can  be  simplified  in  practical  cases  by  assuming 
that  if  a  cause  of  failure  occurs,  then  a  failure  must  occur,  other¬ 
wise  the  cause  must  not  have  occurred.  Therefore,  the  quantity 
P(F/Ci)  is  equal  to  1,  and  the  probability  of  failure  is  equal  to  the 
probability  of  the  cause  occurring,  giving: 

R  =  n  [  1  -  P(C  )]  (3) 

i=  1 
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(Note:  Cases  can  exist  when  a  cause  of  failure  occurs  without 
a  failure  occurring.  For  example,  a  cause  of  failure  can  occur 
in  a  redundant  system  without  the  system  failing.  Such  cases 
are  discussed  in  Chapter  '  !.  For  the  purposes  of  this  fundamental 
discussion,  it  is  assumed  that  a  failure  always  occurs  when  a 
cause  exists.  ) 


The  quantity  P(C-)  can  now  be  re-defined  as  the  "unreliability" 
of  the  item  with  respect  to  failure  cause  C ■ .  Also  substituting 
P(F /C^)  -  1  in  expression  ( 1  ) ,  [1  -  P(Cq)  ]  -  ftp  so  that  Expr e s - 
sion  (3)  can  be  written: 

R  -  n  R-  (4) 

i  —  1 


This  expression  can  be  interpreted  as  stating  that  the  overall 
reliability  of  an  item  is  equal  to  the  product  of  the  probabilities 
that  the  item  will  not  fail  due  to  any  of  the  n  possible  causes  of 
failure,  given  that  if  any  "cause"  exists,  a  failure  will  occur. 

If  the  "causes"  of  system  failure  are  interpreted  as  independent 
failures  of  physical  elements  of  the  sytem  (i.e.,  subsystems, 
equipments,  units,  etc.),  then  Rj  in  expression  (41  can  be  inter¬ 
preted  as  the  reliability  of  the  i™  physical  element  of  the  system 
out  of  a  total  of  n  independent  elements. 

2.2  Reliability  as  a  Function  of  Time 


To  this  point,  reliability  has  been  considered  simply  as  a  proba¬ 
bility  of  success  (no  failure).  However,  reliability  has  been 
defined  with  reference  to  a  "specific  period  of  time."  There¬ 
fore,  the  expression  (4)  will  be  restated  in  terms  of  time  as 
follows : 


(5) 


where: 


R(t)  =  The  probability  that  the  system  will  not  fail  before 
time  t  .  (In  this  case  a  "system"  is  considered  to 
be  any  device  consisting  of  n  elements,  none  of 
which  can  fail  without  system  failure.  ) 

til 

R(t).  =  The  probability  that  the  i  element  of  the  system 
will  not  fail  before  time  t  . 
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This  relation  ship  between  reliability  and  time  provides  the  basic 
mathematical  loundation  of  reliability  prediction,  and  permits 
predictions  to  be  based  on  measurable  time-to-failure  character¬ 
istics  of  system  elements.  The  factor  R(t).  will  now  be  examined 
in  more  detail.  1 

Let: 

R(t) .  __  Probability  of  survival  of  element  i  over  time  t  . 


Then 

f(t). 

1 


dR(t). 

- : - =  The  survival  density  function,  i.  e.  ,  the 

n  t 

probability  that  a  failure  will  not  occur  in 
the  next  time  increment  dt  . 


d  [  1  -  R(t) ,  1  dR(t), 

Now, - - - -  -  -  - - -  -  -  f(t).  t=  The  failure  density 

dt  dt  l - 

function,  i.  e.  ,  the  probability  that  a  failure  will  occur  in  the  next 
time  increment  dt. 


Let: 


z(t).  -  The  hazard  rate,  or  the  probability  that  a  failure 

will  occur  in  the  next  instant  of  time  assuming  pre¬ 
vious  survival,  then: 


(6) 


The  quantity  z(t).  can  be  defined  as  the  hazard  rate  of  element 
i  at  time  t  .  In  general,  it  can  be  assumed  that  the  hazard 
rate  of  electronic  elements  remain  constant  over  practical 
intervals  of  time,  and  that  z(t).  =  \  ;  the  constant,  expected 
number  of  random  failures  per  unit  of  operating  time  of  the  iLn 
element,  i.  e.  ,  the  failure  rate.  Thus,  when  a  constant  failure 
rate  can  be  assumed: 


dR(t). 

dt 

R(t).’ 

i 
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Solving  this  differential  equation  for  R(t)^  gives  the  exponential 
distribution  function  commonly  used  in  reliability  prediction: 


-X  t 

R(t).  =  c  1 

l 

_ 1 


(7) 


Also,  the  mean  time  to  failure  can  be  determined  by: 

MTBF  =  R (t)  dt.  • 

JO 

so  that,  when  a  constant  failure  rate  can  be  assumed: 


MTBF. 

l 


c 


X.t 

l 


(8) 


Expressions  (7)  and  (8)  are  the  basic  mathematical  relation¬ 
ships  used  in  reliability  prediction.  It  must  be  noted,  however, 
that  these  expressions  were  derived  based  on  the  fundamental 
assumption  that  the  failure  rate  of  the  item  under  consideration 
is  a  constant.  When  the  failure  rate  is  not  constant,  the  more 
general  hazard  rate  must  be  considered,  in  which  case  the 
element  reliability  is  obtained  using  the  more  general  expres¬ 
sion: 

r*  co 

-  \  z(t).  dt 

JO  i 

R(t).  -  e  (9) 


The  emphasis  on  the  exponential  distribution  in  reliability  work 
makes  a  discussion  of  the  use  of  this  function  a  s  a  failure - 
probability  model  worthwhile.  The  mechanism  underlying  the 
exponential  reliability  function  is  that  the  hazard  rate  (or  the 
conditional  probability  of  failure  in  an  interval,  given  survival 
at  the  beginning  of  the  interval)  is  independent  of  the  accumulated 
life. 

The  use  of  this  type  of  'failure  law"  for  complex  systems  is 
usually  justified  because  of  the  many  forces  that  can  act  upon 
the  system  and  produce  failure.  For  example,  different 
deterioration  mechanisms,  different  part  hazard-rate  functions, 
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and  varying  environmental  conditions  often  result  in  effectively 
random  system  failures. 

Another  justification  for  assuming  the  exponential  distribution 
in  long-life  complex  systems  is  the  so-called  "approach  to  a 
stable  state,"  wherein  the  system  hazard  rate  is  effectively 
constant  regardless  of  the  failure  pattern  of  individual  parts. 

This  state  results  from  the  mixing  of  part  ages  when  failed 
elements  in  the  system  are  replaced  or  repaired.  Over  a 
period  of  time,  the  system  hazard  rate  oscillates,  but  this 
cyclic  movement  diminishes  in  time  and  approaches  a  stable 
state  with  a  constant  hazard  rate.. 

A  third  justification  for  assuming  the  exponential  distribution 
is  that  the  exponential  can  be  used  as  an  approximation  of  some 
other  function  over  a  particular  interval  of  time  for  which  the 
true  hazard  rate  is  essentially  constant. 

The  preceding  paragraphs  are  not  intended  to  imply  that  the 
exponential  assumption  is  generally  valid.  Because  of  its  mathe¬ 
matical  simplicity  and  the  extensive  theory  developed  by  many 
researchers,  the  exponential  density  plays  a  prominent  role  in 
reliability  work.  However,  if  observed  failure  data  do  not  sup¬ 
port  the  exponential  assumption,  or  if  such  factors  as  wear-out 
are  expected  to  be  significant,  the  exponential  assumption  can 
be  erroneous.  In  such  cases,  other  distributions,  such  as  the 
lognormal,  gamma  and  Weibull  distributions  are  available  for 
performing  more  valid  predictions.  These  more  complex  situ¬ 
ations  will  not  be  treated  in  this  chapter.  For  information  con¬ 
cerning  the  application  of  such  techniques,  the  reader  is  directed 
to  the  references  listed  in  Chapter  12.  Also,  some  discussion 
of  distribution  other  than  exponential  is  presented  in  Chapter  10. 

System  Reliability- Element  Reliability  Relationships 

Expressions  (5),  (7),  and  (8)  are  the  most  common  basic  expres¬ 
sions  used  as  mathematical  foundations  of  reliability  prediction. 
This  is  partly  because  of  the  proportion  of  cases  in  which  a  con¬ 
stant  hazard  rate  or  failure  rate  can  be  assumed,  and  partly 
because  of  the  relative  simplicity  with  which  exponential  functions 
can  be  manipulated.  This  ease  of  manipulation  is  especially 
valuable  when  the  total  system,  subsystem,  or  equipment  relia¬ 
bility  is  being  predicted  as  a  function  of  the  reliability  of  lower- 
level  elements. 


Consider  the  application  of  expression  (5)  in  calculating  the  total 
reliability  as  a  function  of  the  reliabilities  of  individual  elements. 
Substituting  expression  (7)  for  R.(l)  gives: 


n  "V  “  M1 
R(t)  tt  e  -  e  .  e 

i  1 


This  can  be  simplified: 


R(t)  -  e 


-(x  jt+\2t+ ... +\nt)  -(x1  +  \2+ 


+  X  )t 

n 


The  general  form  of  this  expression  can  be  written: 


R(t)  =  e 


Another  important  relationship  is  obtained  by  considering  the 
system  failure  rate  (  \  g)  to  be  equal  to  the  sum  of  the  individual 
failure  rate  of  n  independent  elements  of  the  system  such  that: 


Xs  =  ZXi  . 


Revising  expression  (8)  to  refer  to  the  system  rather  than  an 
individual  element  gives  the  mean  time  between  failures  of  the 
system  as: 


MTBF  = 


PREDICTION  TECHNIQUES 


Typical  reliability  prediction  techniques  within  each  of  the  categories 
defined  in  paragraph  1.  2  are  described  in  subsequent  paragraphs  of 
this  chapter.  These  techniques  are  also  summarized  in  Tables  IX-2 
IX-3  and  IX-4  with  respect  to  factors  that  should  be  considered  in 
selecting  the  most  appropriate  technique  for  a  given  application. 


Table  IX—  l  Reliability  Prediction  Techniques  Useful  During  Detailed  Design  Stages  of  Acquisition  Phase 
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Related  time  (man -months)  is  best  engineering  judgement  of  relative  time  to  obtain  data,  develop  model,  apply  technique  and  prepare  report.  Assumes  a  typical  system  of 
moderate  complexity.  These  values  should  be  considered  in  their  relative  sense  only. 


Table  IX  -2  summarizes  those  techniques  that  are  of  most  va.lue  to 
Air  Force  reliability  specialists  in  performing  feasibility  and  alloca¬ 
tion  studies  during  the  Conceptual  and  early  Definition  1  bases.  Table 
IX-3  summarizes  more  detailed  techniques  that  are  commonU,  used 
by  contractors  in  performing  allocations  and  trade-off's  during  the 
Definition  Phase,  and  in  preparing  input  to  design  reviews  during  the 
Acquisition  Phase.  These  techniques  are  also  used  by  the  SPO  in 
comparing  alternate  designs  and  evaluating  proposals.  The  techni¬ 
ques  summarized  in  Table  IX-4  are  relatively  complex  and  expensive 
in  application,  and  are  usually  used  by  contractors  in  performing 
critical  design  analyses  and  trade-off  studies  during  the  detailed 
engineering  activities  of  the  Acquisition  Phase. 


SIMILAR  EQUIPMENT  TECHNIQUES 

Several  techniques  have  been  developed  and  used  in  performing  very 
early  predictions  of  equipment  reliability  before  any  characteristics 
of  the  system  design  have  been  established.  The  most  basic  of  these 
techniques  involve  a  simple  estimate  of  equipment  reliability  in  terms 
of  MTBF,  failure  rate,  or  similar  parameters,  based  on  experience 
gained  from  operational  equipments  of  similar  function. 

In  general,  these  similar  equipment  prediction  techniques  involve  the 
following  steps: 

a.  Defining  the  new  equipment  in  terms  such  as  general  equipment 
type  (e.g.,  radar),  operational  use  (e,  g.  ,  ground  based)  and  other 
known  characteristics. 

b.  Identifying  an  existing  equipment  or  class  of  equipments  that  most 
nearly  compares  with  the  new  equipment. 

c.  Obtaining  and  analyzing  historical  data  generated  during  operation 
of  the  existing  equipment  to  determine,  as  nearly  as  possible,  the 
reliability  of  the  equipment  under  the  stated  operating  environment. 

d.  Drawing  conclusions  concerning  the  level  of  reliability  that  will  be 
demonstrated  by  the  new  equipment.  Such  conclusions  assume 
that  similar  equipment  will  exhibit  similar  reliability,  and  that 
reliability  achievement  evolves  in  ar,  orderly  manner  from  one 
generation  of  equipments  to  the  next.  These  reliability  prediction 
techniques  permit  very  early  estimation  of  the  failure  rate  of  a 
new  equipment  based  on  experience  gained  from  operational  equip¬ 
ments  of  similar  function.  The  accuracy  of  the  estimates,  how¬ 
ever,  depends  on  the  quality  of  historical  data,  and  the  similarity 
between  the  existing  and  new  equipments. 


Obviously,  more  meaningful  and  accurate  results  are  achieved  if  a 
technique  based  on  field  results  of  similar  products  is  used.  Also, 
other  factors  such  as  design  practices,  and  production  techniques 
are  more  likely  to  be  similar  to  those  on  past  equipments  designed 
and  built  by  the  same  manufacturer  than  those  of  another  manufac¬ 
turer. 

In  most  cases,  prediction  techniques  such  as  this  are  used  in  esti¬ 
mating  the  feasibility  of  meeting  some  minimum  reliability  objective 
within  the  constraints  of  the  current  state-of-the-art. 

5.  SIMILAR  COMPLEXITY  TECHNIQUES 

Several  techniques  are  available  for  performing  reliability  predictions 
based  on  the  complexity  of  the  equipment  of  interest.  These  techni¬ 
ques  liave  been  developed  as  a  result  of  analyses  that  indicate  a  direct 
and  predictable  correlation  between  equipment  complexity  and  relia¬ 
bility.  However,  such  predictions  are  complicated,  somewhat,  by 
the  influence  of  the  equipment  tvpe  or  different  environments  in  which 
the  equipment  will  be  operated.  Therefore,  methods  for  predicting 
reliability  as  a  function  of  equipment  complexity  include  provisions  for 
compensating  for  use  environment  factors. 

The  most  commonly  used  simUar  complexity  techniques  involve  the 
use  of  graphical  procedures  relating  failure  rate  to  active  element 
group  count  and  use  environment.  Two  representative  examples  of 
such  techniques  are  described  below. 

5.1  MIL- STD -7 56 A  AEG  Method 

A  graphical  device  for  relating  reliability  to  electronic  equipment 
complexity  is  provided  in  MIL-STD-756A .  Application  of  the  pro¬ 
cedure  involves  estimating  the  number  of  active  element  groups 
(AEG's)  in  each  functional  block  of  the  equipment.  An  active  element 
is  defined  as  an  electron  tube  or  transistor,  except  that  ten  com¬ 
puter  diodes  and  associated  circuitry  are  considered  as  an  active 
element  in  digital  computers.  This  graph  should  not  be  used  for 
equipment  containing  integrated  circuits. 

The  graph  published  in  MIL-STD-756A  for  determining  reliability 
in  terms  of  feasible  mean  life  or  MTBF  is  reproduced  in  Figure  9-1 
This  graph  includes  two  bands  indicating  the  probable  range  of 
achievable  reliability  for  equipments  to  be  operated  in  airborne  and 
shipboard  environments.  The  higher  MTBF  values  for  a  given  number 
of  series  active  elements  represent  the  level  of  reliability  that  can 
be  achieved  with  good  reliability  engineering  and  design  effort. 
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FEASIBLE  MEAN  LIFE  -  MTBF  IN  HOURS 


The  reliability  estimate  obtained  from  this  chart  represents  a  band  of  possible  outcomes  The  smaller 
failure  rate  values  of  the  band  arc  obtainable  with  good  reliability  engineering  and  design  effort. 
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NUMBER  OF  SERIES  ACTIVE  ELEMENTS  -  N 

(Reproduced  from  Ml L-STD-756A) 

Figure  9-1  MTBF  Versus  Functional  Complexity  for  Electronic  Equipment 
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5.2  Bird  Engineering-Research  Associates  Method 

Devices  similar  to  the  MIL-STD--756A  AEG  method  described  in 
paragraph  5.  1  have  been  developed  for  other  operational  environ¬ 
ments,  and  are  available  in  a  variety  of  sources.  Two  of  these,  as 
developed  by  Bird  Engineering-Research  Associated,  Inc.,  are  pre¬ 
sented  in  Figures  9-2  and  9-3.*  Procedures  for  using  these 
methods  are  essentially  the  same  as  for  the  MIL-STD- 756a  method. 
However,  the  source  document  also  describes  the  following  procedure 
for  estimating  functional  complexities  (number  of  AEG's  in  an  equip¬ 
ment  function).  These  procedures  do  not  apply  for  equipments  con¬ 
taining  integrated  circuits. 

Functional  complexity  is  related  to  the  number  of  series  "active" 
elements  (transistors,  diodes,  tubes,  etc.)  to  be  used  in  the  sub¬ 
system.  Detailed  schematics  for  the  new  design  are  normally 
not  available  in  the  early  stages  of  system  planning.  It  may  be 
necessary,  therefore,  to  estimate  complexity  of  the  electronic 
subsystem  on  the  basis  of  previous  designs  of  comparable  func¬ 
tional  complexity  and  performance  requirements. 

Table  IX-5  tabulates  the  range  of  series  complexity  (in  AEG's)  in 
which  predecessor  designs  have  fallen.  In  the  absence  of  a  better 
estimate  of  AEG  complexity  for  specific  functional  blocks  within 
the  particular  subsystem  under  study,  an  estimate  may  be  derived 
from  the  table  for  preliminary  feasibility  estimation. 

In  those  instances  in  which  logic  diagrams  and  circuit  schematics 
are  available  for  certain  blocks  of  the  functional  block  diagram, 
the  AEG  density  per  block  is  estimated  by  making  an  actual  count 
of  the  active  elements.  When  the  subsystem  under  consideration 
is  predominantly  analog  in  function  and  is  comprised  of  200  or 
more  active  element  groups  of  all  types,  it  is  permissible  to  give 
all  AEG's  equal  weight  -  i.  e.  ,  1AEG  =1.0  unit  of  complexity. 

However,  if  an  estimate  of  AEG  complexity  is  to  be  made  on  a 
component-by- component  basis ,  where  individual  component 
complexities  are  considerably  less  than  200,  a  more  realistic 
appraisal  of  functional  complexity  is  obtained  by  applying  a  weight¬ 
ing  factor  to  each  class  of  active  elements  in  the  component.  A 
list  of  "relative"  weighting  factors  applicable  to  electronic  compon¬ 
ents  is  shown  in  Table  IX- 6  . 

Once  the  complexities  have  been  estimated,  the  nomograph  of 
Figure  9-2  can  be  used  to  estimate  average  failure  rates  as  a 
function  of  complexity  within  analog  subsystems,  to  account  for 
catastrophic  as  well  as  tolerance  and  interaction  failures.  The 
following  rules  and  assumptions  apply  to  analyses  made  at  the 
subsystem  level: 

Handbook  of  Reliability  Prediction  Procedures  for  Ballistic  and  Space  Systems , 

Bird  Engineering-Research  Associates ,  Inc.,  Vienna,  Va,  DDC  No.  AD470294. 
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EQUIVALENT  ANALOG  FUNCTIONAL  COMPLEXITY 
N  ACTIVE  ELEMENTS 

(Reproduced  from  Handbook  of  Reliability  Prediction  Procedures  for  Ballistic  and  Space  Systems 
Bird  Engineering-Research  Associates,  Inc..,  DDC  No.  AD470294-) 

Figure  9-2.  Reliability  Feasibility  Estimation  Nomograph  for  Analog  Electronic  Functions 
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Figure  9-3.  Reliability  Feasibility  Estimation  Nomograph  for  Digital  Electronic  Functions 
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Table  lx-5.  AEG  Complexities  Required  for  t lie  Pcrlormance  of  Specific  Electronic/ 
Electrical  Subsystem  functions  in  Ballistic  and  Space  Systems 


A 

Analog  AF.G  Range 

Subsystem  aim  Function 

Minimum 

1 

Maximum 

Guidance  and  Control: 

Command  Receiver  (basic) 

no 

100 

Guidance  Computer  (digital) 

500  ( d) 

1500  (d) 

Programmer  Logic  &  Switching  (digital) 

50  (d) 

250  (d) 

Signal  Processor 

2 

10 

Analog  Servo  Amplifier 

4 

10 

Inertial  Reference  Platform 

50 

BOO 

I  .orizon  Sensor 

20 

100 

Star  Tracker 

20 

100 

Gyro  Package 

10 

50 

Electrical  System: 

Battery  Supply  &  Temp.  Control  Ckts 

10 

60 

DC/AC  Inverter,  Single-Phase 

20 

50 

DC/AC  Inverter,  3-Phase 

40 

1 

100 

AC/DC  Ccnverter/Regulator 

5 

20 

Telemetry: 

Transmitter 

10 

30 

Modulator 

5 

50 

Signal  Processor  (per  channel) 

5 

10 

Sensor/Amplifier 

2 

5 

Receiver 

20 

60 

Beacon  Transponder 

40 

100 

Arming  and  Fuzing 

20 

200 

Sense/Switch  Device  (for  redundant  designs) 

5 

50 

$ 

All  values  are  equivalent  analog  AEG’s  for  use  with  Figure  9 

-2  except  those  denoted  by 

|  (d)  which  should  be  used  with  Figure  9-3. 
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Table  IX -6.  Weiuhting  Factors  for  Estimating  Equivalent  Analog  Complexity  of 
Electronic  Subsystems  (for  Use  With  Figure  9-2; 


AEG  Type 

Function 

Equivalent 

Analog 

AEG .  s 

Transistor  or 

Signal -level  analog  function 

1.  0 

Electron  Tube 

Signal-level  digital  function 

Power  conversion  and  regulation 

0.  1 

Diodes 

Signal -level  analog  function- 

0.  1 

Signal -level  digital  function 

0.01 

Microwave 

Traveling  wave  tubes, 

Powei  Tubes 

magnetrons,  klystrons 

100.  0 

Photo-Electric  Cell 

Light  sensor  functions 

0.  1 

Photo  Multiplier 

Light  amplifier 

10.0 

Solar  Cell 

Power  generation 

0.01 

Relays 

General 

1.  0 

Gyros,  Position 

Inertial  reference 

50.  0* 

Gyros,  Rare 

"Rare"  signal 

10.0 

Accelerometers 

Acceleration  measurement 

1.0 

Crystals 

Frequency  determination 

1.  0 

6 

For  short -duration  missions  —  e.  g. ,  ballistic  flight  and  space  flights  of  less  than  500 
hours. 


I 


a.  A  power  subsystem  of  a  given  complexity  is  assigned  an 
AEG  failure  rate  twice  that  of  an  AEG  in  an  analog  subsystem 
of  the  same  complexity. 

b.  The  number  of  digital  AEG's  in  an  analog  subsystem  should 
be  divided  by  ten  if  on3y  the  analog  AEG  chart  (Figure  9  -2) 
is  used  for  subsystem  failure-rate  estimation. 

For  digital  AEG's  in  a  digital  system  -  e.  g.  ,  computers  -  the 
nomograph  presented  in  Figure  9-3  should  be  used.  This 
family  of  nomographs  relates  failure  rate  of  digital  computer 
functions  to  series  complexity  of  the  digital  system.  Digital 
system  failure  -  rate  estimates  made  on  the  basis  of  these  nomo¬ 
graphs  usually  fall  within  +200%  and  -67%  of  the  values  actually 
observed  in  the  field.  Digital  system  failure-rate  estimates 
must  be  considered  tentative  until  verified  by  test. 

6.  PREDICTION  BY  FUNCTION  TECHNIQUES 

Recently,  several  techniques  have  been  developed  which  permit  the 
prediction  of  equipment  or  system  reliability  with  relation  to  the  func¬ 
tional  characteristics  of  the  equipment,  as  opposed  to  complexity  or 
part  stress /population  techniques  normally  employed.  These  techni¬ 
ques  are  based  on  correlations  between  significant  functional  character¬ 
istics  and  observed  operational  reliability.  These  techniques  are  not 
intended  to  replace  conventional  methods  of  reliability  prediction,  but 
rather  supplement  these  techniques  by  permitting  reliability  predic¬ 
tions  during  early  phases  of  a  design  cycle  when  data  concerning  part 
application  are  not  available. 

Three  typical  reliability  prediction  by  function  techniques  arc  le scribed 
below.  For  the  purpose  of  this  notebook,  these  techniques  will  be 
identified  as  follows: 

a.  FEC  Ground  Equipment  Method.  This  technique  is  described  in 
Technical  Report  No.  RADC-TR-65- 27,  System  Reliability  Pre¬ 
diction  By  Function,  May  1 9 U  DDC  No.  AD466025. 


b.  ARINC  Ground  Equipment  Method.  This  technique  is  described 
in  Technical  Report  No.  RADC -T  R- 63- 300 ,  System  Reliability 
Prediction  By  Function,  which  includes: 

Volume  I  -  Development  of  Prediction  Techniques, 

DDC  No.  AD416494 

Volume  II-  Prediction  Procedure,  DDC  No.  AD418192. 
Supplement  1,  Revised  Equations, 

DDC  No.  AD6 1 4227 

c.  ARINC  Airborne  Equipment  Method.  This  technique  is  described 
in  Technical  Report  No.  RADC -T R- 66 - 509,  Avionics  Reliability 
and  Maintainability  Prediction  by  Function.  DDD  No.  AD802998. 

6.  1  FEC  Ground  Equipment  Method 

This  technique  makes  use  of  a  set  of  equations  and  equivalent 
graphs  relating  to  ground-based  equipment  function  (e.  g, ,  receiver, 
transmitter,  etc.  )  and  a  major  operational  characteristic  of  that 
function  (e.g.,  noise  figure,  peak  power,  etc.)  to  a  predicted 
value  of  MTBF  or  failure  rate.  The  total  system  reliability  can 
be  estimated  by  appropriately  combining  individual  function  relia¬ 
bilities. 

Predictions  are  performed  as  follows: 

a.  Identify  the  equipment  according  to  one  of  the  following  general 
functions  as  most  applicable: 

.  Radar  Receiver /Transmitter 
.  Radar  Display 
.  Radar  Receiver 
.  Radar  Transmitter 
.  Communications  Receiver 
.  Communications  Transmitter 
.  Communication  Multiplex 
,  EDP  Central  Processor 
.  EDP  Peripheral  Equipment 

b.  Determine  values  of  key  parameters  associated  with  equipment 
in  question.  Key  parameters  are  identified  in  Table  IX -7  for 
respective  equipment  functions. 

c.  Determine  the  equipment  MTBF  or  Failure  Rate  using  either  the 
equations  presented  in  Table  IX-7  ,  or  the  referenced  graphs  as 
appiopiiate. 
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Table  XX- 7.  Parameters  and  Equations  For  FEC  Prediction  By  Function 
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FIGURE  9-6  RADAR  RECEIVER  FUNCTION  -  NORMALIZED  FAILURE  RATE  VS.  PEAK  POWER 


FIGURE  9-7  RADAR  TRANSMITTER  FUNCTION  -  NORMALIZED  FAILURE  RATE  VS.  PEAK  POWER 
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FIGURE  9-9  COMMUNICATION  TRANSMITTER  FUNCTION- 
MTBF  VS-  POWER  GAIN 
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FIGURE  9-11  EDP  CENTRAL  PROCESSOR  FUNCTION 
MTBF  VS.  WORD  SIZE/ADD  TIME 


Table  IX-8.  Reliability  Data  For  EDP  Peripheral  Equipment 


Function 

Type 

Failure  Rate 

6 

(Failure  Per  10  hours) 

05%  Confidence  Band 

Around  Failure  Rate 

Input 

Paper  Tape  Reader 

113 

37 -278 

Output 

Paper  Tape  Punch 

136 

5C  -297 

Typewriter 

779 

389-1394 

Print  Unit 

3413 

2662-4326 

Input/Output 

Magnetic  Tape  Drive. 

786 

710  -866 

Typewriter 

446 

289 • 860 

Auxiliary 

Drum 

2137 

1030  -2764 

Storage 

Buffer 

Tape  Control  Unit 

976 

794-1192 

Regression  equations  for  EDP  peripheral  equipment  could  not  be  developed.  These  poiir.  - 
values  for  typical  equipments  can  be  used  where  appropriate.  If  other  peripheral  equipment 
are  used,  reliability  data  should  be  obtained  from  other  sources. 


d.  When  appropriate,  determine  system  MTBF  by  combining 

equipment  MTBF s  or  failure  rates  as  indicated  in  Table  IX-  7. 

6.  2  AR1NC  Ground  Equipment  Method 

This  technique  utilizes  a  set  of  "prediction  equations"  relating 
mean  time  between  failure  (MTBF)  to  the  simultaneous  influences 
of  a  variety  of  parameters  characterizing  the  equipment  of  interest 
Predictions  are  performed  as  follows: 

a.  A  prediction  equation  is  selected  such  that  the  associated  pre¬ 
diction  parameters  most  nearly,  correspond  to  the  character¬ 
istics  of  the  equipment  under  consideration.  The  equations 
and  the  respective  prediction  parameters  are  listed  in 

Table  IX-9. 

b.  Prediction  parameters  for  the  seiecced  equation  are  quantified 
in  accordance  with  the  instructions  in  Table  IX  10. 

c.  The  prediction  equation  is  solved  using  these  parameter  values. 
Initial  solution  of  the  equation  provides  a  predicted  value  of 

Ln  g,  where  Q  is  the  predicted  mean  time  between  failures. 

The  predicted  §  can  then  be  determined  from  the  expression. 


6  =  e 


Ln  0 


d.  The  confidence  intervals  of  the  predicted  values  are  computed 
using  the  worksheets  in  Figures  9-12,  9-  13,  or  9- -14.  The 

worksheet  selected  should  correspond  to  the  particular  predic¬ 
tion  equation  used.  The  approximate  95%  confidence  interval 
around  the  predicted  value  of  9  is  given  by: 


A 


£  e  s  eg 


K 

where  g  =  e  ,  and  K  is  calculated  using  the  worksheet. 


6.  3  ARINC  Airborne  Equipment  Methods 

This  technique  is  basically  an  extension  of  the  ARINC  Ground 
Equipment  Method  described  in  paragraph  6.  2  to  permit  pre¬ 
diction  of  avionics  equipment  reliability.  This  technique  is 
somewhat  more  comprehensive  in  scope,  and  permits  con¬ 
sideration  of  the  function  of  the  line  replaceable  unit  (LRU). 


9-37 


Table  IX -  9  Ground  Equipment  Reliability  Prediction  Equations 


Table  IX -10.  Prediction  Parameters 


(Extracted  from  RAlx'-TDR-u" -300,  Supplement  1) 


(Extracted  from  RADC-TDR-63-300 ,  Supplemer”-  1) 


Independent 
V  ariables 
X. 


X  = 
a 


X  = 
c 


Mean  of 
Independent 

V  ariables 

X. 

l 

Deviation 
(X.  -  X.) 

i  l 

5.  274 

1.  560 

1.911 

(a) 

F  actor  1 


(b) 

Factor  2 


(c) 

Multiplier 


(Xa  -  Xa)  = 
(Xb  -  xb)  = 
(Xc  -  Xc)  = 

(Xa  -  xa)  = 
(xa  -  Xa)  = 
(Xb  -  xb)  = 


0.  032 
0.  080 
0.  004 
-0.  022 
-0. 006 
0.  021 


(Extracted  From  RADC-TDR-fi3-300 

Supplement  i) 


SUBTOTAL 

ADD' 

Constant  value 

f=TOTAL 
s  = 


K  =  2  (  V~f  )  (s) 


g  -  e 


1.  020 


0.400 


Figure  9  -12  Worksheet  for  Computing  Confidence  Interval  for  Equation  A 


Mean  of 

Independent 

Independent 

Variables 

Variables 

Xi 

Xi 

Xd  = 

0.  247 

Xe  ^ 

2.  733 

Deviation 
(Xi  -  Xi  ) 


(a) 

<b) 

(c) 

F  actor  1 

F actor  2 

Multiplier 

<Xd  -Xd)= 

<Xd  -Xd)= 

0.  0‘>7 

(xe  -  x  )  = 

(Xe  -Xe>= 

0.  010 

<Xd  -Xd)= 

<Xe  -Xe)= 

0.  032 

SUBTOTAL 


ADD 

Constant  Value 


DTOTAL 


Product 
a)  (b)  (c 


1. 037 


K  =  2  (J~T )  ( s )  = 


g  =  eK  = 


0.  460 


as  well  as  the  overall  function  of  the  equipment.  The  LRU  level 
prediction  will  not  be  described  here,  however,  because  to  do 
so  would  require  excessive  space.1" 

The  technique  for  predicting  airborne  equipment  reliability  in¬ 
cludes  the  following  steps: 

a.  The  general  type  of  equation  to  be  used  is  determined  based 
on  the  general  type  of  information  available.  Two  types  of 
equations  are  defined  for  reliability  prediction  at  the  equip¬ 
ment  level.  These  are: 

Type  I  -  General  mission/performance  parameters  are 

known,  as  during  the  Definition  Phase  of  system 
development. 

Type  ii  -  Detailed  performance/design  parameters  are 

defined,  as  during  the  early  Acquisition  Phase. 

b.  The  appropriate  equipment  classification  is  determined  based 
on  the  definitions  in  Table  IX- 1 1 .  This  table  also  includes  a 
description  of  the  sample  that  was  used  in  developing  the  techni¬ 
que.  These  descriptions  can  be  used  in  determining  the  extent 
to  which  the  equipment  under  consideration  compares  with  the 
sample  equipments. 

c.  The  appropriate  prediction  equation  and  the  ranges  of  predic¬ 
tion  parameters  are  selected  using  Table  IX- 12.  The  equations 
for  the  "all  equipments"  class  should  be  used  with  caution  and 
only  if  one  or  more  of  the  parameter  ranges  of  the  appropriate 
class  are  exceeded. 

d.  The  prediction  parameters  applicable  to  the  selected  prediction 
equation  are  identified  and  quantified  using  Table  IX- 13. 

e.  The  confidence  interval  for  the  reliability  prediction  is  determined 
using  the  following  equations: 

•fcn  eL(  =  £n  e  -  k;  eL  =  (e"k)6 
f  n 9  =  £n  §  +  k;  0TT  -  (ek)  0 

J  u  J  u 


For  a  detailed  description  of  the  technique  for  considering  the  LRU  in 
predicting  reliability  by  function,  the  reader  is  referred  to  the  references 
cited  in  paragraph  6c. 
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Claaslf  ication 

Navigational  and  Radio 
Receiving  Seta 


I  ahlC  1X-11  Equipment  Classifications 


Determines  navi  gat  loniil  parameters  by  the 
receiving  and  prooeBalng  of  signal  a  rrom 
external  oour^ea  or  by  the  receiving  and 
detecting  of  various  signals  used  In  con¬ 
nection  with  air  traffic  control  and  land- 
liig  aids. 


Description  of  Sample 

The  sample  represent  e.  If  eqjlpmenta  with 
a  range  In  I-flUr'  of  from  13  to  2975  hours. 
All  receivers  are  mock  mounted;  only  onu 
contains  printed  circuits;  no  redundancy 
Is  Incorporated,  and  both  tubed  and  tran¬ 
sistorized  sets  are  well  represented.  The 
receivers  consist  of  marker  beacon,  direc¬ 
tion  flr.der,  Loran,  glide  slope,  data  link, 
localizer,  and  Sonobouy  receivers. 


Electromechanical  Analog 
Navigational  Computers 


Accepts  electrical  Input  signals  from  a 
variety  of  aircraft  sources  such  as  static 
pilot  tubes,  Joppler  information,  gyro¬ 
scopes,  and  compass  systems.  The  computer 
Integrates  these  Inputs  into  useful  navi¬ 
gational  information  for  display  to  the 
aircraft's  crew.  These  computers  contain 
no  electromagnetic  transmitting  or  receiv¬ 
ing  functions. 


The  sample  is  characterized  by  these  com¬ 
puters  's  use  of  servo  systems  to  solve 
simple  trlgoi.oretric  relations.  This  group 
consists  of  computers.  Including  Air  Dats 
Computers,  True  Airspeed,  Inertial  Naviga¬ 
tional,  and  Target  Positioning  Computers. 
MTBF  ranges  from  14  to  1)13  hours.  All  but 
2  of  the  computers  were  transistorized  and 
all  were  analog  In  operation. 


Indicator  Qroups 


Displays  processed  information  from  a 
variety  of  on-board  equipments;  corslsts  of 
self-contained  devices,  but  this  class  dees 
not  include  those  Indicators  classified  a3 
pilot's  instruments  and  used  as  flying  aids. 


The  sample  Is  derived  from  3  equipments 
with  fTTBF's  ranging  from  60  to  162  hours. 
Coordinate  data  nts  and  PPI  displays  are 
represented.  7  :  gh  observed  correlation 

Is  partially  a  rto.lt  of  the  small  sample 
size . 


Signal  Processing/Gener¬ 
ating  Equipment 


Radio  Command  Communi¬ 
cations 


Performs  the  intermediate  function  of  gener¬ 
ating  or  processing  signals  for  display  or 
for  use  In  other  equipments.  The  class  con¬ 
sists  of  coder/decodera,  signal  analyzers, 
and  signal  converters. 


Provides  air-ground  or  alr-alr  radio  communi¬ 
cations  receiving  and  transmitting  sets. 


The  sample  represents  only  4  equipments, 
throe  of  which  perform  distinctly  different 
functions;  yet  all  are  within  the  classifi¬ 
cation  of  signal  processing  or  generating. 
The  equipments  are  the  Signal -Data  Converter, 
Signal  Analyzer,  and  Coder-Decoder.  The  KT5F 
for  these  equipments  ranges  from  29  to  41*^ 
hours.  The  high  observed  correlation  is 
partially  because  of  the  small  sample  size. 


The  sample  represents  1?  equipments  of  the 
AN /ARC  series.  The  equipments  use  CW,  AM, 

FM,  and  5SB  emission  In  frequency  ranges  of 
from  9  to  400  me.  All  sets  that  use  SSB 
modulation  have  a  power  output  of  ICO  watts 
or  more,  whereas  the  sets  using  CW,  AM,  FM, 
have  power  outputs  ranging  from  2  to  50  watts. 
The  MTBF  of  these  equipments  ranged  from  8  to 
521  hours.  Only  or.e  transistorized  equipment 
Is  in  this  group;  the  remain!'  .  sets  U3e 
standard,  miniature,  and  subnlnlature  tubes; 
75#  of  the  sets  provide  fully  automatic 
tuning.  Only  one  set  Is  manually  turned. 


High-Power  Radar  Sets 


Provides  high  effective  peak  output  power 
(5  KV  or  greater). 


The  sample  represents  15  radar  nets  which 
have  on  I-7TBF  range  of  from  5  to  50  hours. 
Most  of  the  parameters  observed  have  well 
distributed  values  throughout  their  respec¬ 
tive  ranges  for  all  fifteen  radar  sets  ana¬ 
lyzed.  Care  should  be  used  In  determining 
the  values  of  the  significant  variables  for 
tils  class.  Particular  caution  should  be 
taken  In  the  determination  of  operational 
functions  other  than  those  listed  as  repre¬ 
sentative  of  this  parameter. 


Lo*-;-'.wer  Navigational 
and  IFF  Transmitting 
and  Receiving  Seta 


Trovides  relatively  low-power  outputs  (leas 
than  5  KW  effective  peak  power),  and  all 
equipments  in  this  class  perform  both  a 
receiving  and  transmitting  function.  This 
class  Includes  doppler,  beacons,  7ACAN, 
altimeters,  and  IFF. 


This  sample  represents  22  Bets,  the  largest 
group.  It  also  represents  the  widest  variety 
of  equipment  types,  Including  doppler,  beacons, 
TACAN,  altireters,  and  IFP .  The  F7TBF  for  this 
group  ranges  from  14  to  459  hours.  One  altim¬ 
eter  utilized  an  analog  system  at  low  altitudes 
and  piloed  system  at  high  altltvides.  Frequen¬ 
cies  range  from  about  1  to  30  KKHz. 


I  n  t  e  re  annum  1  c  at  1  on 
Gets 


Provides  communications  between  on-board 
aircraft  stations;  also  works  In  conjunction 
with  on-board  radio  receiving  and  transmit¬ 
ting  sets. 


This  sample  represents  6  equipments,  with  a 
range  In  ITIjF  of  from  *40  to  000  hours.  Two  of 
the  six  equipments  studied  use  ■  lcuum  tubes; 
the  remaining  four  equipments  are  transistor¬ 
ized.  The  equipments  can  accommodate  from  4 
to  17  channels .  Some  of  the  seta  provide  their 
own  power,  the  others  use  the  aircraft's  power 
directly,  The  high  observed  correlation  is 
partially  because  of  the  email  sample  size. 


(Extracted  from  RADOTR-66-509) 
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'1  able  IX-12  Avionics  Equipment  Reliability  Prediction  Equations 

E^ulpiwPt  Classification 

Equation 

Type 

Equation 

Para<>  eter 

Range 

Havlf.it lonal  and  Radio 

Receiving  Sets 

I 

irv 0  •  5.796-0.56l(X1)-n.336(x?)*0.232(£n  Xj) 

0.1  <  Xl  »  4.5 

0  *  x2  *  4 

2  *  X3  *  500 

II 

2n8  -  3.5?9-0.612(X1)-0.395(Xj)t0.130(x^) 

0.1  «  x1  «  9.5 

0  ‘  X2  «  9 

13  ‘  X„  «  22 

FK*ctrcr.echanlcal  Analog 
Navigational  Computera 

I  4  II 

InS  »  !. 806-9. 120(X5)40.298(Xf  ) 

3  *  x5  *  20 

9  *  X6  i  13 

I  ’.dlcators 

II 

Jn8  •  5.515-u.ltj(X7) 

2  «  X7  *  8 

Signal  Procesalng/Oeneratlng 
Equipment 

I  4  II 

in 9  -  6.283-0.0176(X13) 

20  t  X13  ‘  168 

Radio  Command  Communications 

I  4  II 

9  -  8. 779-0. 708(2nXli()-0. 359(2  nX15) 

9  s  a  ■» J l) 

2  *  X]5  s  400 

H’.gf  Jower  Radar  Seta 

{ 

I 

ir.9  ,  3.317-0.267(Xl6)-0.136(X17)*0.291(Xlg) 

1  »  xl6  ‘  “ 

2  *  X„  •  9 

2  .  Xl6  1  9 

II 

ins  -  9. 16I1-0. 323(Xl6)-0.270(in  Xj) 

1  *  Xl6  .  9 

3  ‘  X7  *  185 

Low  Power  Navigation  &  IFF 
Transmitting  4  Receiving 

Seta 

I  4  II 

me  .  ii.3ii9-o.mi5(x2i.o.350(xa) 

0  ‘  %  ‘3 

0  ‘  X8  ‘3 

Ir tsrcomnur.lcatlon  Seta 

'  I 

ms  «  6.973-l.?i3(X1)-1.159(X9) 

0.1  *  X;  *  2.0 

4  <  Xq  «  17 

II 

2n8  -  7.108-0.0202(X10)-0.507(X1) 

20  *  X1Q  *  140 

0.1  *  Xx  *  2.0 

All  Equipments 

I 

InS  .■  2.986<0.2lt2(X11)-0.2?6(  ir.  X^ 

-0. 112(X2)+0 .0999C Xg) 

1  *  *  10 

0.1  *  Xl  t  48.7 

0  *  Xj  Ml 

5  4  X6  *  13 

II 

2n§  .  9.707-0. 19l(2n  X 10 ) *0 . 1 83 1 Xjj) 

-0.993(in 

19  l  X10  ‘  11,000 

1  «  Xn  *  10 

9  ‘  x12  «  579 

8  «  X„  ‘  22 

(Extracted  from  RADC-TR-GG-509) 
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Table  IX -13  Avionic  Equipment  Parameters  and  Quantification 


Parameter 

Symbols 


Humber  of  Interfacing  Equipment# 


Packaging  Characterlet tc  Rating 
Characteristic : 

Type  of  Enclosure 

Some  cabinet#  preaaurlzed 
No  cabinet#  pressurized 
Vibration  Isolation 

Some  cabinet#  shock  mounted 
Ho  cabinets  ahock  mounted 
Equipment  Packaging 

Equipment  In  single  package 
Equipment  In  3  to  *»  packagca 
Equipment  In  b  to  8  package# 

Equipment  in  9  or  more  package# 

Type  of  Cooling 

Forced  air-refrigerated  (at  all  time#) 
Forced  alr-inslde  ambient  (at  all  time#) 
Convection 

Refrigerated  air  on' deck,  outalde 
ambient  at  altitude 

Component  Packaging 
Modularized 

Conventional  Construction 

Type  of  Wiring 
Printed  Circuit# 

Conventions*  W'.rlng 


Number  of  Signal  Input#  Acceptable 


Equipment  Feature  Rating 
Peaturei 

Power  Supply 

Power  supply  external  to  equlpmer  ; 
Solid  State 

Combination  aolld  state  and  tube# 
Tube 

Rotating  machinery 
Tuning  (Operational) 

None  required 
Manual 

Semi-automatic  (auto-tune) 

Fully  automatic 
Type  of  Indicator# 

None 

Meter# 

Electro-mechanical 
Cathode  ray  tube 


Number  of  LRU 'a 

LRU  Function  (KIL-STD-196A  Symbol): 

Air  conditioning  (HD)  Indicator,  Non  C.R.T.  (ID) 
Amplifier  (AM)  Junction  Box  (J) 

Antenna,  conplex  (AS)  Xeyer  (KY) 

Antenr.s,  almple  (AT)  Power  Supply  (PP) 

Compensator  (CN)  Receiver  (R) 

Computer  (CP)  ReeeiverAransmitter  (RT) 

Control  |C)  Recorder  (RD) 

Converter  (cv)  R»lay  Bex  (RE) 

Coupler  (CTI)  Switch  (SA) 

Indicator,  Cathode  Transmitter  (T) 

Ray  Tube  (IP) 


Equipment  Suhfunctlon  rating  for  Lew  Power  Navigation 
and  IPP  Transmitting  and  Receiving  Seta. 

Sub function:  Rating 

Doppler .  TACJN,  Radio  Altimeter*  1 

Beacons  2 

IPP  Set#  3 


Quantification 


Equipment  volume  In  cublo  feet 

Note:  For  Radio  and  Navigational  Receiving 
Seta,  tha  volume  of  any  antenna  which  may  b« 
a  part  of  the  cat  la  not  lnoluded  In  the 
calculation. 


The  number  of  other  equlpmento,  excluding  indicators, 
thrt  feed  signals  to  or  receive  algnela  from  this 
equipment . 


Measured  In  u  volts  for  a  10 


Tha  sum  of  the  ratings  given  to  each  characteristic 
for  the  equipment  under  study. 


The  number  of  signal  a  from  other  equipmenta  which  the 
equipment  under  etudy  requires  or  can  accomodate  In 
its  operation. 


The  aura  of  rating#  for  the  applicable  Individual 
design  feature*. 


The  aura  of  the  total  quantity  of  each  VAV  Included 
In  the  equipment  complement. 


The  rating  of  the  aubfunctlon.  t 
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Parameter 
Symbol  a 


Quantification 


The  number  of  channeli  of  operation  for  Inter¬ 
communication  seta. 


T>ie  steady  state  power  In  watts  consumed  ty  the  equip¬ 
ment  in  iL*i  most  power-oonsumlng  mode  of  operation. 
Note:  Considers  the  "rtdlete"  not  the  "standby" 
status  of  rB.dar  nets,  "steady  state"  Implies 
that  starting  power  requirements  are  not  to 
be  considered. 


equipment  Function  Rating 
Classification  Function:  P„a 

Navigational  Receiving  Seta 

S/*?an 

fcyclo  Receiving  Sets 
Radio  Navigation  Seta 
Direction  Finder  Equipment 

Electromechanical  Analog  Navigation  Computers 
Navigation  Computers 

Indicator  Oroup 
Indicators 

Signal  Proceeslng/Oeneratlne  Equipment 
Signal  Converter 
Signal  Analyzers 

Coder -Decoder  1 

Radio  Command  Communlcat  ions 
Radio  Command  Communl cat  Ions 

High-Power  Radar  Seta 
Intercept 
Tracking 
Side -Looking 
S^#rch 

Plre  Control 

Bcmblng/Navlgallon 

Acquisition 

Low  Power  Navigational  and  IFF  Transmitting 
1(1(1  Receiving  Setc 
IFF 

Doppler 

Beacone 

TACAN 

Altimeters 

Xnttrcoramunlcatlons 
Intercom  Sets 


X  -1 

a12 

LRU  Rat 

i-vg 

The 

sum  of  the 

products  of  the  quantity  of 

each  of 

LPU  i 

unction  (Kli. 

-STB-196A  Symbol):* 

the 

LRU 

LRU  types 
typo. 

shown  below  times  the  rating 

for  that 

Ratln#, 

Rsllnj 

HD 

2 

ID 

' 

AM 

3 

J 

3 

AS 

5 

KY 

k 

AT 

1 

PP 

u 

CN 

2 

R 

h 

CP 

U 

RT 

k 

C 

3 

RO 

U 

CV 

3 

RE 

li 

cu 

1 

SA 

2 

IP 

k 

* 

Power  Output 


Number  cf  Operational  Functions  or  Capabilities 
Radar  Set  Functions: 


Tne  weight  in  pounds  of  the  equipment. 

Note:  P.  r  Radio  Navigations  1  end  Receiving 
Sets,  the  weight  of  any  entenna  that  may  be 
a  part  of  the  set,  Is  excluded  from  the 
calculation. 


The  highest  Irequency  of  operation  In  megacycles  per 
second. 


The  power  In  watts  delivered  to  the  antenna. 


The  total  number  of  the  functions  contained  by  the 
radar  aet: 


*  See  3^ 


Airborne  Early  Warning 
and  Control 
Anti -Intrusion 
Acquire  on  Jammer 
Acquisition 
Bombing 

Calibrating  (Test  Equipment) 
CW  Illumination 
ECM  Training 

Oround  Controlled  Approach 
Oun  Plre  Control 
Home  or.  Jammer 
Height  Finding  Radar 
IF P/S I? 


for  definition*  of  aymbols. 


Intercept 
Navigation 
Nolae  Jamming 
Projectile  Intercept 
Radar  Decoy 
Radar  Beacon 
Radar  Trainer 
Ranging 

Reconnaissance 

3earch 

Track 

Any  functions  not  other¬ 
wise  llated  of  the  eamc 
order 
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I  aMo  IX-  )■>  (Ontmued) 


Para/noter 
3ymbol a 

Parameter 

Quantification 

*17 

Number  of  feature* 

Feature ii 

KTI 

Multiple  Hi.njee 

SCCM 

Self  contained  aonputer 

Self  contained  display 

Variable  pules  xldth 

Beam  ahaplna  (1.0. >  ccoecuint  equaled) 

Variable  Scan  characteristics 

Variable  range  bug 

Self  a ontainad  gyro 

Contains  boaaon  receiver 

The  aum  of  *.he  number  of  the  f naturae  contained  by  the 
radar  eat. 

* 

% 

X 

! 

CD 

Type  of  Active,  Element  Oroup  /AEO) 

Typ«  of  *m  f  Predooi  runt  J  rutin* 

*>.•:  rating  for  *r.*  ’.>pi*  of  AFO  u*.d. 

Old  dVofdQe  ci '  rnllngr.  If  more  than  one 
type  doninates. 

Tranelator  t* 

Tube,  Standard  or  miniature  3 

Tube,  Quboiniature  2 

Electromechanical  devices  1 

(Extracted  from  RADC-TR-GG-oOfl) 


where 


9  and  @  are  the  lower  and  upper  confidence  limits, 
l-i  U 

spectively,  and 


re  - 


k  = 


^a/^d)3 


j  =  i+l 


r-  1 

C .  .  X.  X  . 

—  1J  J 
i=l 


where 


m  =  number  of  observations  used  in  the  regression  analysis 
r  =  number  of  independent  variables 

t.  ,  ,  d)  =  t  statistic  for  a  100  (1  -  a)  %  two-sided  confidence 
2  interval  based  o  i  d  =  (m-r-1)  degrees  of  freedom 


standard  error  of  estimate 


c„  =  Gauss  multiplier 

_ 

x.  =  Deviation  (X^  -  X^)  where  X^  is  the  value  of  the  i 

independent  variable  tc  be  used  for  the  prediction  and 
X^  is  the  mean  value  of  the  i^  independent  variable  of 
the  equipments  used  in  the  regression  analysis. 

If  one-sided  limits  are  desired,  the  t  statistic  t(r,,d)is  used 
to  compute  k;  for  a  one-sided  lower  limit,  subtract  k  from 
Pn  c  ;  for  a  one- sided  upper  limit,  add  k.  Table  IX  -  1  4  presents 
the  information  required  to  compute  k  .  The  t  statistic  shown 
in  the  table  is  for  a  two-sided  90%  confidence  interval  or  a 
one-sided  95%  confidence  interval.  Also  shown  in  Table  IX- 14 
is  the  sample  multiple  correlation  coefficient,  R,  for  each 
equation. 


7.  PART  COUNT  PREDICTION  TECHNIQUES 

Reliability  prediction  techniques  are  available  for  application  when 
data  such  as  estimated  part  inventories  are  available,  but  when  the 
design  details  necessary  for  applying  the  more  detailed  stress  analysis 
techniques  are  not  known.  These  techniques  make  vise  of  "average" 
failure  rates  for  general  classes  and  types  of  parts,  based  on  data 
acquired  from  field  experience  on  a  large  variety  of  equipment.  The 
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Table  IX -H  Factor;  for  Determining  Confidence  Intervals  for  Prediction  Equations 


Equatii  n 

Equipment 

t 

3 

m 

Variable  1 

Gauss  Multi- 

Number 

Classification 

(for  90‘S  C.L.  ) 

1 

-  Code 

Xi 

pllerB 

(Early)'- 

Radio  and  Navi¬ 
gational  Receiving 

0.800 

1.796 

0.792 

15 

1 

X1 

1.345 

cn=  0.0510 

Sets 

2 

X2 

1.652 

c1£»  0.00783 
c22=  0.0359 
c13=  0.0210 
c23-  0.000795 

3 

in  X3 

1.763 

c33=  0.0550 

2 

Radio  and  Navi- 

0.804 

1.796 

0.785 

15 

l 

X1 

1.345 

cn=  0.0442 

(Late ) 

gatlonal  Receiving 

Sets 

2 

X2 

1.652 

c12=  0.00733 
c22=  0.0359 

3 

X4 

18.427 

C^3=  0.00506 
c23=-0. 000792 
c33=  0.0202 

3 

Electromechanical 

0.824 

1.796 

0.601 

14 

l 

xs 

10.147 

cn=  0.00172 

(Early 

Analog 

and 

Late ) 

Navigational  Com¬ 
puters 

2 

x6 

10.480 

c12=-0. 00426 
c22=  0.127 

4 

(Late ) 

Indicator  Groups 

0.938 

2.353 

0.166 

5 

1 

*7 

5.398 

C1X=  0.0442 

5 

Signal  Processing/ 

0.962 

2.920 

0.453 

4 

1 

x13 

72.2 

o11«  0.0000603 

(Early 

Generating  Equip- 

and 

Late ) 

ment 

6 

Radio  Command 

0.719 

1.833 

O.761 

12 

1 

in  Xl4 

5.177 

°11=  0.0937 

(Early 

and 

Late ) 

Communications 

2 

in  Xlg 

3.420 

e 12=  0.0596 

c2?=  0.0798 

7 

(Early) 

High  Power  Radar 
Sets 

0.753 

1.796 

0.401 

15 

1 

xl6 

2.632 

cu=  0.1/8 

2 

x17 

6.161 

c12=-0.0221 
c22=  0.0274 

3 

X18 

2.626 

e13=  0.0582 
c23- -0.0378 

C33.  0.0252 

8 

(Late ) 

High  Power  Radar 
Sets 

0.766 

1.782 

0.375 

15 

1 

X16 

2.632 

c11=  0.102 

2 

In  Xj 

2.841 

c12=-0.0221 

Table  IX-14  Continued 


Equation 

Number 

Equipment 

Classification 

R 

t 

(for  90%  C.L.) 

s 

m 

Variable 
|  -  Code 

y 

*1 

Gauts  Multi¬ 
pliers 

9 

( Ea  rly 
and 
Late) 

Low  Power  Naviga¬ 
tion  and  IFF 
Transmitting  and 
Receiving  Sets 

0.750 

1.729 

0.610 

22 

1  x2 

2  Xq 

1.720 

1.418 

cxl=  0.041 9 
c12=  0.0206 
c22=  0.0845 

10 

Intercommunication 

Sets 

0.971 

2.353 

0.517 

6 

>■'/  1 

cn=  0.611 
o12=  0.0419 

c22=  0.0122 

11 

(Late ) 

Intercommunication 

Sets 

1 

2.353 

0.407 

6 

1  X10 

2  *1 

38.708 

0.546 

C]L1=  0.00029C 
Ci2=-0. 00448 
c22-  0.536 

12 

All  Equipments 

0.657 

1.645 

0.921 

94 

1  X11 

2  In  Xx 

3  X2 

4  X6 

1.892 

0.850 

2.888 

8.653 

cxl-  0.00413 
c12=  0.00038? 
c13=  0.000318 
cl4=-0. 00132 
c22=  0.00802 
c23=-0. 00156 
o24=  0.00197 
c33=  0.00240 
c34=-0. 00103 
c44=  0.00394 

13 

(Late) 

All  Equipments 

0.735 

1.645 

0.828 

94 

1  In  X1Q 

2  Xn 

3  X12 

4  x4 

6.432 

1.892 

3.0Q4 

15.019 

cn=  0.0173 
c12=  0.00429 
c13=-o,.oi33 
cl4=  0.00099c 
c22-  0.00512 
c23=-0. 00231 

cg4=- O.OOO525 
0^  =  0.0234 
c34=  0.000228 
c44«  0.00120 

(Extracted  from  RADC-TR-G6-509.) 
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technique  described  below  is  typical  of  currently  used  part  count 
prediction  technique s. 

This  tecluiique  permits  estimating  equiptneni  failure  rate  based  on 
part  count  (actual  or  estimated)  by  class  or  type.  The  technique  in¬ 
volves  counting  the  number  of  parts  of  each  class  or  type,  multiplying 
this  number  by  the  generic  failure  rate  for  each  part  class  or  type, 
and  summing  these  products  to  obtain  the  failure  rate  for  the  equipment. 
The  procedure  distinguishes  a  part  class  as  being  all  parts  of  a  given 
function  («.  g.  ,  resistors,  capacitors,  transformers).  Part  types  are 
used  to  further  define  parts  within  a  class  (e.g.,  fixed  composition 
resistors,  fixed  wire  wound  resistors). 

Table  IX-  ]  5  provides  average  failure  rate  values  for  a  variety  of  parts, 
by  class  and  type.  These  values  were  derived  from  Volume  II  of  this 
notebook  by  assuming  a  fixed  ground  environment,  operation  at  50°C, 
and  50%  stress  ratio.  The  values  have  been  calculated  for  "lower- grade 
parts,  i.  e.  ,  standard  military  parts  bought  to  lesser  MIL- specification 
and  without  requiring  any  special  reliability  controls  (see  page  3  of 
Volume  II).  Other  modifying  factors,  such  as  resistance  factor,  con¬ 
struction  class  factor,  etc.,  were  assumed  as  appropriate  based  on 
engineering  judgment. 

The  part  count  prediction  is  performed  as  follows : 

a.  All  parts  in  the  equipment  under  consideration  are  classified  accord 
ing  to  the  classes  and  types  listed  in  Table  IX- 15. 

b.  The  total  number  of  parts  in  each  class  and  type  is  determined  by 
actual  count  if  possible.  II  an  actual  count  is  not  possible,  the  best 
estimate  of  part  count  is  obtained. 

c.  The  average  failure  rate  is  obtained  from  Table  IX- 15  for  each  type 
of  part  used  in  the  equipment. 

d.  The  equipment  failure  rate  is  determined  using  the  expression 

n 

*  e  =  I NA 

i=l 

where: 

\  =  The  equipment  failure  rate 

N.  =  The  number  of  parts  of  type  i  included  in  the  equipment 
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TABLE  TX-I5.  Average  Part  Type  Failure  Rates 


Part  Class  and  Type 

Failure  Rate* 

(%  per  1,000  hours) 

CAPACITORS 

MIL- C- 25, 

Paper  Foil 

.  00  l 

MIL-C-  141  57, 

Paper,  Paper  Mylar 

.  001 

MIL  -  C  -  18312, 

Mylar  -  Metallized 

.  001 

MIL-C-  1 9978, 

Mylar  or  Teflon 

.  036 

MIL-C-  1 9978, 

Polystyrene 

.  041 

MIL-  C- 27287, 

Plastic  Film 

.  0038 

MIL-C-  .1965, 

Tantalum  Foil 

.  045 

MIL-C-  3965, 

Tantalum,  Wet  Slug 

.  032 

MIL-C-26655, 

Solid  Tantalum 

.  0024 

MIL-  C-  39003, 

Solid  Tantalum 

.  0024 

MIL-C-62, 

Aluminum,  Wet  Foil 

.  142 

MIL-  C-  5, 

Mica,  Molded 

.  0018 

MIL-C-5, 

Mica,  Dipped 

. 00086 

MIL-  C-  5, 

Mica,  Button 

.  063 

MIL-C-  1 1  272, 

Glass  &  Porcelain  Enamel 

.  0032 

MIL-C- 20, 

Ceramic,  Low  K 

.  0322 

MIL-C-1]  015, 

Ceramic,  High  K 

.018 

MIL-C- 81, 

Variable,  Ceramic 

.  568 

MIL-C- 14409, 

Variable,  Glass  Piston 

.  039 

Table  IX -1  5  (cent  d) 


1 

Part  Class  and  Type  j 

Failure  Rate* 

%  per  1 , 000  hours) 

RESISTORS 

MIL-R  - 1 1, 

Carbon  Composition 

. 00207 

MIL-R-  11804, 

Power  Film 

.  160 

MIL-R-  10509, 

Precision  Film 

.  0015 

MIL-R-  55  1  82, 

Fixed  Established  Rel.  Film 
(R  el.  Level  R) 

•  00006 

MIL-R-22684, 

Insulated  Fixed  Film 

.  0186 

MIL-R-  93, 

Accurate  Wirewound  Fixed 

.  0397 

MIL-R-26, 

Power  Wirewound  Resistors 

.  0390 

MIL-R-  12934, 

Precision  Wirewound  Potentiometer 

.  50C 

MIL-R-  1 9, 

Semi-precision  Wirewound  Pot. 

.  235 

MIL-R-  94, 

Low  Precision  Composition  Pot. 

.  700 

MIL-R-22097, 

Non- Wirewound  Trimmer  Pot. 

.  630 

MIL-R-27208, 

Wirewound  Trimmer  Pot. 

.  099 

MIL-R-3901  5, 

Established  Rel.  Wirewound  Pot. 

(R  el .  Level  R ) 

,0126 

MIL-R-  22, 

High  Power  Wirewound  Pot. 

.  168 

CONNECTORS 

,  Grade  C  (per  mated  pair) 

.  0324 

RELAYS, 

General  Purpose  DPDT 

.  072 

SWITCHES, 

Snap-action  DPDT 

.  450 

MOTORS,  Case  B,  grade  1,  split-phase  .0153 


i 


Table  IX- 1  5  (cont 'd) 


Part  Class  and  Type 

Failure  Rate* 

(%  per  1,000  h<uur s) 

FANS  &  BLOWERS 

6.  3 

SYNCHROS  &  RESOLVERS 

.  080 

AUDIO  TRANSFORMERS 

.  0038 

MAGNETIC  AMPLIFIERS  (  <100v) 

.  0075 

POWER  TRANSFORMERS  &  FILTERS,  Low-level 

. 00625 

MICRO- SIZED  MAGNETIC  DEVICES,  All  Types 

.  075 

RF  TRANSFORMERS  &  COILS 

.  0938 

LOW-LEVEL  PULSE  TRANSFORMERS 

.  0019 

DIODES 

Logic  Switching 

.  023 

Power  Rectifier 

.  110 

TRANSISTORS 

Analog,  Silicon,  npn 

cc 

CM 

Digital,  Silicon,  npn 

.  039 

MICROCIRCUITS 

Digital,  Average  Grade 

.  0840 

Linear,  Average  Grade 

.  2520 

TUBES 

Receiving  Tubes,  See  Table  Xr  3,  page  295  of  Volume 

II 

Transmitting  Tubes,  See  Table  X-7,  page  303  of  Volume  II 

Special  Purpose  Tubes,  See  Table  X-8,  page  306  of  Volume  II 

*NOTE:  These  failure  rates  were  derived  from  Volume  II  of  the  RADC 

Reliability  Notebook,  assuming  a  fixed  ground  environment, 

operation  at  50  q  and  a  50%  stress  ratio,  and  are  for  "lower 
grade"  parts. 
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-  The  average  failure  rate  of  parts  of  type  i 

n  =  The  number  of  different  types  of  parts  included  in  the 
equipment 


8.  STRESS  ANALYSIS  PREDICTION  TECHNIQUES 

Several  reliability  prediction  techniques  have  been  developed  that  per¬ 
mit  a  detailed  part-by-part  analysis  of  a  system  design  to  the  extent 
that  the  effects  of  degrading  stresses  are  considered  in  determining 
the  failure  rates  of  individual  parts.  These  techniques  are  all  similar 
in  application,  the  major  difference  being  the  source  of  failure  rate 
data,  and  the  corresponding  differences  in  the  procedures  used  in  ex¬ 
tracting  data  from  the  data  source,  and  translating  these  data  for 
application  to  a  specific  system.  Once  the  failure  rate  data  are  ob¬ 
tained,  the  reliability  prediction  is  completed  by  combining  the  failure 
rates  for  each  part  in  the  system  according  to  a  pre-established  mathe¬ 
matical  model.  In  general,  this  will  involve  substituting  failure  rates 
for  the  values  ir  expressions  such  as  equations  (10)  and  (11)  of  para¬ 
graph  2.  3  of  this  chapter  to  obtain  the  predicted  reliability  or  MTBF 
of  an  element  of  the  system,  and  combining  system  element  reliabilities 
as  appropriate  to  obtain  a  prediction  of  the  overall  system  reliability. 

Volume  LI  of  this  notebook  is  a  recommended  source  of  failure  rate 
data  suitable  for  application  in  performing  stress  analysis  prediction 
techniques.  Failure  rate  data  and  related  information  are  organized 
by  part  class  (i.  e.  ,  resistors,  potentiometers,  etc.)  and  type  or  style 
(e.g.,  composition  resistor,  film  resistors  and  wirewound  resistors). 

The  procedure  for  extracting  failure  rate  data  from  Volume  II  differs 
according  to  part  class  and  type.  In  general,  however,  the  following 
steps  are  required: 

a.  Operational  and  environmental  stresses  and  other  characteristics 
are  determined  for  each  part  in  the  subject  equipment  or  system. 
The  specific  types  of  parameters  and  characteristics  that  must  be 
defined  and  evaluated  varies  with  part  class.  The  extent  of  this 
step  of  the  procedure  is  indicated  by  Figure  9  -15,  which  lists  the 
general  types  of  parameters  and  characteristics  that  must  be  con¬ 
sidered  in  determining  failure  rates  for  parts  in  each  class. 

b.  A  base  failure  rate  is  determined  for  each  part.  This  value  is 
established  from  the  appropriate  chart  in  Volume  H,  and  is  a 
function  of  part  type,  environmental  temperature,  and  the  relative 
level  of  the  more  significant  operational  stress. 
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c.  rill  values  of  one  or  more  multiplicative  or  additive  factors  are 
determiner,  from  tables  or  charts  in  Volume  II,  These  factors 
define  the  relationship  between  the  base  failure  rate  and  the  pre¬ 
dicted  failure  rate  for  the  specific  application  of  interest. 

d.  The  part  failure  rate  is  calculated  using  the  established  base 
lailure  rate  and  the  modifying  factor.  Equations  for  performing 
these  calculations  are  provided  in  Volume  II. 

Stress  analysis  failure  rate  predictions  such  as  this  permit  extremely 
detailed  analyses  of  equipment  or  system  reliability.  However,  since 
details  of  the  system  design  are  required  in  determining  stress  ratios, 
temperature  and  other  application  and  environmental  data,  these 
techniques  are  only  appli  able  during  the  late  stages  of  design.  Also 
because  of  the  high  level  of  complexity  of  modern  systems,  the  appli¬ 
cation  of  the  procedure  is  time  consuming,  and  should  only  be  used 
when  such  detailed  part-by-part  analysis  is  warranted.  Computer 
processing  of  reliability  data  is  normally  feasible.  However,  the 
initial  stress  analysis  usually  involves  considerable  effort  at  the  engi¬ 
neering  level. 

DEGRADATION  ANALYSIS  PREDICTION  TECHNIQUES 

Degradation  analysis  is  a  general  term  defining  a  variety  of  analysis 
techniques  which  permit  an  evaluation  of  the  probable  susceptibility  ol' 
an  equipment  to  variation  or  drift  of  circuit  parameters.  In  genera;, 
degradation  analysis  is  performed  using  any  of  a  number  of  circuit 
design  analysis  techniques.  These  techniques  are  basically  similar  in 
that  they  involve  a  computer  solution  of  a  mathematical  model  describ¬ 
ing  circuit  output  variables  in  terms  of  the  several  interrelated 
input  parameters,  The  results  of  a  degradation  analysis  can 
provide  information  concerning  the  relationships  between  input  and 
output  parameter  variation  or  drift,  and  can  have  valuable  application 
in  the  solution  of  a  variety  of  reliability  design  problems  such  as 
achieving  maximum  circuit  stability  within  specified  performance  con¬ 
straints,  or  identifying  most  likely  causes  of  failure. 

Many  proven  circuit  analysis  techniques  suitable  for  degradation 
analysis  are  currently  available  for  use.  Some  techniques  are  more 
applicable  to  circuit  design  while  others  are  effective  in  predicting 
circuit  performance  under  a  range  of  conditions.  The  former  type  is 
generally  useful  in  optimizing  a  circuit  design  in  terms  of  reliability, 
while  the  latter  type  can  be  used  to  provide  a  measure  of  circuit  relia¬ 
bility  under  a  variety  of  operational  environments,  or  after  a  given 
period  of  service.  Some  techniques  can  only  be  used  on  linear  cir¬ 
cuits  while  others  may  be  used  on  nonlinear  circuits  as  well.  Some 
are  intended  for  static  analysis  of  d.  c.  or  a,  r.  circuits  while  others 


can  be  used  for  dynamic  transient  analyses.  Also,  input  data  require¬ 
ments  range  from  simple  nominal  and  limiting  v<  '•  >  s  to  complete 
statistical  distributions  of  parameter  values. 

Because  of  the  large  number  and  complexity  of  analysis  techniques 
available,  a  detailed  discussion  of  degradation  analysis  is  not  within 
the  scor.-e  of  this  notebook.  However,  many  of  the  available  techniques 
can  be  dentified  as  being  developed  from,  or  similar  to,  one  of  the  four 
general  methods  discussed  below.  These  methods  are  categorized  for 
the  purposes  of  this  discussion  as  the  "parameter  variation",  "worst 
case",  'moment",  and  "Monte  Carlo1' methods  of  circuit  analysis.  The 
following  discussions  are  presented  to  aid  m  identifying  a  particular 
technique  with  reference  to  one  of  these  general  methods.  In  addition, 
significant  characteristics  of  each  method  are  summarized  in  Table 
IX- 16  to  aid  in  selecting  an  appropriate  method  for  a  given  application. 

9.  1  Parameter  Variation  Methods 

Parameter  variation  methods  can  provide  information  concern¬ 
ing  part  parameter  drift  stability,  circuit  performance-part 
parameter  relationships,  and  certain  circuit-generated  stresses. 
These  methods  require  the  least  amount  of  input  information  of 
the  four  methods  discussed  here,  and  do  not  require  extensive 
data  from  large  sample  life  tests  on  circuit  elements.  In  gen¬ 
eral,  input  data  consists  of  nominal  or  mean  values  and  esti¬ 
mated  or  assumed  circuit  parameter  drift  characteristics. 

In  the  general  case,  a  parameter  variation  analysis  would  con¬ 
sist  of  solving  circuit  equations  in  an  iterative  manner  until  a 
solution  is  obtained  for  all  probable  combinations  of  input 
parameter  values.  This  general  solution,  however,  would 
usually  involve  an  excessively  large  number  of  solutions,  and 
would  be  impractical  in  most  applications.  In  view  of  this, 
typical  parameter  variation  analyses  are  performed  by  varying 
each  parameter  independently  (one-at-a-time)  or  in  pairs 
(two-at-a-time) . 

A  typical  parameter  variation  method  would  be  performed  by 
varying  one  or  two  parameters  at  a  cime  in  discrete  steps  with 
all  other  parameters  held  at  the  nominal  value.  Solution  of  the 
circuit  equations  for  each  step  .vou.ld  provide  the  limiting  values 
of  the  particular  parameters  under  investigation.  This  procedure 
is  repeated  for  all  parameters  or  pairs  of  parameters  in  the  circuit. 

Output  data  from  an  analysis  of  this  type  could  include: 

a.  Parameter  limits  outside  of  which  a  failure  would  occur. 

b.  Maximum  stress  levels  within  the  circuit,  such  as  maximum 
power  dissipated  by  a  resistor  or  transistor,  breakdown  volt¬ 
ages,  maximum  or  minimum  biasing  voltages  or  current. 
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c.  Most  critical  circuit  parameters  with  respect  to  causing 
circuit  failure. 

9.  2  Worst-Case  Methods 

Worst-case  methods  of  circuit  analyses  can  be  used  to  determine 
the  worst-case  conditions  for  any  output  variable  of  the  circuit. 

Once  the  worst- case  conditions  have  been  determined  for  any  output 
variable,  the  value  of  that  output  variable  is  calculated  and  com¬ 
pared  with  the  rated  or  specified  value.  The  worst-case  methods 
are  extensions  of  the  parameter  variation  methods  in  that  output 
variables  are  evaluated  with  reference  to  varying  values  of  the 
input  parameters.  The  major  difference  is  that  all  parameters  are 
simultaneously  varied,  with  each  parameter  being  varied  in  the 
direction  that  produces  an  increased  (or  decreased)  output.  When 
each  parameter  value  has  reached  the  limit  of  its  range  (tolerance 
or  expected  drift  limit)  the  worst-case  output  value  is  determined. 

The  worst-case  method  is  somewhat  more  complex  than  the  param¬ 
eter  variation  method  in  that  the  computer  program  first  evaluates 
partial  derivatives  of  the  circuit  equations  to  determine  the  direc¬ 
tion  each  parameter  should  be  varied  in  reaching  the  worst-case 
condition.  The  circuit  equations  are  then  solved  as  the  parameter 
values  are  varied  in  the  appropriate  direction.  As  the  parameters 
reach  the  limiting  values,  the  value  of  the  output  variable  is  calcu¬ 
lated  and  recorded.  Upon  conclusion  of  the  analysis,  the  program 
summarizes  the  results  by  listing  the  input  parameters,  output 
variables,  and  appropriate  statements  pertaining  to  the  subsequent 
use  of  the  data. 

9 .  3  Moment  Methods 

Moment  methods  of  circuit  analysis  bear  this  name  because  they 
make  use  of  the  mean  and  the  second  moment  (variance)  of  the 
frequency  distribution  of  input  parameters  to  calculate  the  mean 
and  variance  of  the  circuit  output  variables.  Assuming  that  the 
parameter  distributions  are  normal,  these  quantities  would  com¬ 
pletely  describe  the  distribution  of  the  output  variable.  Although 
distributions  are  rarely  exactly  normal,  the  results  of  an  analysis 
usually  provide  meaningful  approximation  of  the  circuit  character¬ 
istics. 

The  moment  method  analysis  begins  with  the  preparation  of  a  set 
of  simultaneous  equations  describing  the  circuit.  The  mean  values 
for  the  circuit  output  variables  are  calculated  using  the  mean  values 
for  each  circuit  parameter.  The  computer  then  calculates  the 
variance  of  each  output  variable  by  means  of  the  propogation  of 
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variance  formula.  This  statistical  formula  relates  the  variance  of 
output  variables  to  the  mean,  variance  and  correlation  coefficients 
of  input  parameters. 

Data  inputs  required  for  performing  a  moment  analysis  include  a 
complete  schematic  diagram  of  the  circuit  being  studied,  together 
with  nominal  value  s,  tolerances,  and  characteristics  of  all  com¬ 
ponent  parts.  In  addition,  tabulations  of  the  mean,  variance,  and 

correlation  coefficients  of  all  pertinent  part  parameters  are  re-  '  . 

quired. 

The  outputs  obtained  from  this  type,  of  analysis  include  the  mean 
and  variance  of  all  output  valuables.  In  addition,  the  outputs  can 
include  voltages  at  nodes  of  the  network,  branch  currents,  resistor 
and  transistor  power  dissipations,  and  other  special  variables  of 
particular  interest  to  the  reliability  engineer. 

9.  4  Monte  Carlo  Methods 
— 

Monte  Carlo  methods  of  network  analysis  involve  the  computer 
simulation  of  an  empirical  approach  whereby  a  number  of  copies 
of  the  network  under  study  are  constructed  by  sampling  component 
parts  at  random  from  a  representative  populatic  of  these  parts. 

The  empirical  approach  would  require  the  process  to  be  repeated 
many  times,  and  measurements  made  and  tabulated  for  all  vari¬ 
ables  of  interest  for  each  copy  of  the  network.  It  is  obvious  that 
such  an  approach  would  be  excessively  expensive  and  time  con¬ 
suming. 

A  computer  simulation  of  this  technique  removes  many  of  the  un¬ 
desirable  features  of  an  empirical  approach.  Such  a  simulation, 
known  as  the  Monte  Carlo  method,  is  related  to  other  computerized 
circuit  analysis  methods  in  that  it,  begins  with  a  mathematical  model 
consisting  of  a  set  of  circuit  equations.  However,  the  input  data 
and  computer  time  requirements  for  a  Monte  Carlo  analysis  are 
more  severe  than  for  other  methods.  However,  these  undesirable 
features  are  offset  by  the  more  detailed  output  information  as  com¬ 
pared  to  the  other  methods. 

Input  data  for  a  Monte  Carlo  analysis  must  include  the  complete 
frequency  distribution  of  each  input  parameter.  For  example,  if 
a  resistance  value  is  required,  then  a  set  of  values,  distributed 
in  a  manner  that  is  representative  of  the  actual  resistance  values, 
would  be  provided.  Random  selection  of  a  value  from  this  set 
would  then  simulate  the  random  selection  of  a  resistor  in  the 
empirical  approach, 

(  i 


Selecting  all  input  parameter  values  in  this  manner,  and  solving 
the  circuit  equations  using  the  parameter  values  selected,  will 
result  in  one  set  of  output  variable  values.  Subsequent  repetition 
of  this  process  will  provide  a  number  of  such  sets,  the  distribution 
of  which  will  simulate  the  distributions  that  would  have  been  ob¬ 
served  using  the  empirical  approach.  These  distributions  typically 
are  presented  in  the  form  of  a  set  of  histograms  or  equivalent 
tabular  data  describing  the  distributions  associated  with  each  out¬ 
put  variable.  Thus,  a  Monte  Carlo  analysis  will  provide  the  most 
meaningful  data  concerning  the  expected  performance  of  the  circuit 
under  investigation. 
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CHAPTER  10 


RELIABILITY  MEASUREMENT 

1.  INTRODUCTION 

The  objective  of  reliability  measurement  is  to  obtain  an  empirical 
assessment  of  the  degree  to  which  an  equipment  or  system  meets 
the  reliability  design  requirements.  Information  on  failures  which 
occur  during  a  test  period  in  which  equipments  or  components  are 
subject  to  specified  stresses  provides  data  which  can  be  used  to  com¬ 
pute  component  failure  distributions  or  equipment  reliability  char- 
acterstics  at  spec'fied  levels  of  confidence.  In  this  manner,  the 
buyer  of  components  or  equipment  obtains  a  numerical  indication 
of  the  risk  taken  in  accepting  a  production  lot  of  specific  size  on 
the  basis  of  reliability  data  obtained  from  reliability  tests  per¬ 
formed  over  a  relatively  short  duration. 

Modern  reliability  measurement  procedures  permit  verification  of 
compliance  with  specific  reliability  requirements  by  means  of  spec¬ 
ified  testing  procedures  which  can  be  imposed  as  a  contractual  re¬ 
quirement.  Thus,  reliability  tests  are  practical  tools  which  can  be 
used  to  assure  that  appropriately  reliable  equipments  are  being  pro¬ 
cured. 

The  Air  Force  requires  that  testing  procedures  be  used  to  demonstrate 
the  reliability  of  specific  equipments  and  systems  prior  to  their  accept¬ 
ance  into  inventory.  Systems  are  normally  tested  in  three  phases:  the 
Category  I  Test,  the  Category  II  Test,  and  the  Category  III  test. 

During  the  Category  I  Tests  specific  subsystem  and  equipments  (CEI's) 
are  usually  tested  by  the  contractor  to  determine  if  they  meet  the  re¬ 
liability  requirements  specified  in  the  Part  I  Detail  Specifications. 
Equipments  are  evaluated  in  a  controlled  environment  at  the  contrac¬ 
tor's  plant,  and  performance  is  monitored  by  means  of  an  instrumented 
test  setup.  Equipments  that  do  not  meet  reliability  requirements  must 
be  redesigned.  However,  when  the  mean-time-between-failures  are 
high  and  testing  will  take  a  long  time,  the  reliability  demonstration 
tests  of  CEI's  is  done  during  system  test. 

In  Category  IT  Testing  the  individual  equipments  are  mated  and  the 
entire  system  is  subjected  to  realistic  operational  procedures  and  en¬ 
vironment.  These  tests  are  typically  a  joint  contractor / Air  Force 
responsibility,  and  can  either  be  performed  in  a  simulated  environ¬ 
ment  at  the  system  contractor's  plant,  or  in  an  actual  field  environ¬ 
ment.  These  tests  verify  the  overall  system  reliability,  and  the 
compatibilitv  of  subsystems  and  total  system  operational  effectiveness. 
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At  this  time  the  government  either  accepts  or  rejects  the  system 
and  its  subsystems. 

Category  ill  testing  or  operational  testing  is  supported  by  the 
SPO,  but  is  performed  by  Air  Force;  using  command  personnel, 
who  exercise  the  total  system  in  its  actual  operational  environment. 
Reliability  data  are  gathered  along  with  many  other  types  of  opera¬ 
tional  data.  The  Category  III  Tests  provide  indications  as  to  the 
reliability  performance  of  Air  Force  Systems  in  realistic  operating 
environments  -  these  tests  may  uncover  weaknesses  masked  in  the 
previous  tests. 

In  addition  to  these  equipment  and  system  reliability  tests,  individual 
parts  and  components  also  are.  subject  to  reliability  testing  when  they 
are  produced  in  production  quantities.  In  this  case,  a  sample  of  items 
d '.'awn  from  a  production  lot  is  tested  and  the  test  results  are  used  to 
decide  on  accepting  or  rejecting  the  entire  production  run  at  some 
specified  level  of  confidence. 

The  techniques  of  mathematical  statistics  are  used  extensively  in  re¬ 
liability  test  and  demonstration.  These  techniques  provide  the  tools 
to  relate  sample  size,  test  duration,  confidence  levels,  stress  levels 
and  other  factors  related  to  reliability  demonstration. 

Some  of  the  basic  statistical  concepts  used  in  reliability  measurement, 
are  summarized  below,  followed  by  a  brief  elementary  discussion  of 
Bayesian  statistics  as  applicable  to  reliability  evaluation.  Also,  re¬ 
lationships  between  certain  statistical  parameters  that  must  be  con¬ 
sidered  in  reliability  testing  plans  for  equipments  or  systems,  are 
presented  together  with  procedures  to  aid  in  the  utilization  of  MIL-STD- 
781B  in  the  design  of  reliability  test  plans.  The  chapter  is  concluded 
with  a  brief  discussion  of  certain  considerations  in  the  application  of 
MIL-STD- 1 05 D  in  designing  part  reliability  test  plans. 


2.  STATISTICAL  DISTRIBUTION  OF  RELIABILITY  DATA 

Reliability  engineers  make  use  of  probability  distributions  to  describe 
from  the  data  the  time  to  failure  characteristics  of  components  and 
equipments  and  use  these  statistical  models  to  predict  system  reli¬ 
ability.  Failure  data  are  also  utilized  in  conjunction  with  reliability 
test  and  demonstration  procedures.  Evaluation  of  such  procedures 
is  made  by  fitting  the  failure  data  to  an  assumed  underlying  distribution 
and  determining  whether  the  test  criteria  are  satisfied.  Therefore, 
a  basic  understanding  of  the  use  of  statistical  time  to  failure  distri¬ 
butions  is  fundamental  to  the  development  of  any  reliability  measure¬ 
ment  program.  In  view  of  this,  some  typical  distributions  which 
appear  frequently  in  the  reliability  literature,  including  the  important 
exponential,  Weibull,  and  gamma  distributions  are  reviewed  below. 
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2.  1  Time  To  Failure  Density  and  Distribution  Functions. 

A  central  concept  in  reliability  theory  is  that  of  a  failure  density 
function  or  failure  distribution  function.  These  are  analogous  to 
the  probability  density  function  and  probability  distribution  func¬ 
tion  encountered  in  probability  theory.  The  density  function  mea¬ 
sures  the  probability  that  a  variable  having  a  specific  value  will 
occur.  The  distribution  function  measures  the  probability  that  a 
chance  occurrence  will  fall  within  a  given  range  of  values. 

The  density  function  can  be  illustrated  by  considering  a  discrete 
random  variable,  where  the  variable  can  assume  only  a  given 
number  of  discrete  values.  The  k  possible  values  of  the  variable 
can  be  denoted  by  x^,  x^,  . x^.  For  a  specific  value  of  x, 

for  example,  x^,  the  probability  that  a  random  trial  will  yield  the 

value  x^  is  represented  by  ^x^).  This  is  the  value  of  the  "density 

function,  "  f(x)  at  x  -  x  ,  and  represents  the  relative  number  of 

times  the  value  x^  occurs  out  of  all  x's  in  the  population.  Pictori- 

ally,  a  density  function  for  a  discrete  variable  can  be  represented 
by  a  histogram  such  as  that  shown  in  Figure  10-1.  The  height  of 
the  bars  represents  the  relative  number  of  times  each  value  of  x 
occurs.  Tim  value  f(x  )  is  represented  by  the  height  of  x  bar. 

La  La 

The  density  function  of  a  continuous  variable,  i.  e.  ,  a  variable 
that  can  take  on  any  value,  can  be  represented  by  the  familiar 
"bell"  curve  such  as  that  shown  in  Figure  10-2.  The  height  of  the 
curve  at  a  given  point  on  the  x  axis  represents  the  value  of  the  den¬ 
sity  function  for  that  value  of  the  variable.  This  curve  will  take 
on  a  variety  of  shapes,  depending  on  the  characteristics  of  the 
particular  density  function  under  consideration. 

The  distribution  function,  i.  e.  ,  the  function  describing  the  proba¬ 
bility  that  a  variable  value  falls  within  a  given  range  of  values, 
effectively  measures  the  total  number  of  times  values  of  the  vari¬ 
able  within  a  given  range  occur  in  relation  to  the  total  number  of 
all  possible  values  of  the  variable.  In  the  discrete  case,  the  dis¬ 
tribution  function  is  defined  as  the  sum  of  all  f(x)  values  for  values 
of  x  falling  within  the  range  of  interest.  Mathematically,  the 
distribution  function  for  a  discrete  variable  is  given  by: 

n 

F(x)  =£  f(x.) 
i=m 
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where: 


m  and  n  identify  the  values  of  x  bounding  the  range  of  interest 
and  f(x.)  is  the  relative  density  function;  i.  e.  ,  f(x.)  represents 

the  ratio  of  the  total  number  of  elements  having  a  value  x.  to 

the  total  number  of  elements  in  the  population. 

In  the  continuous  case,  the  distribution  function  is  defined  as  the 
integral: 

F(x)  =  \  k  f(x)  dx 

J 

a 

where: 

f(x)  is  a  density  function  such  as  that  represented  by  the  curve 
in  Figure  10-2  and 

a  and  b  are  the  li.rn  .ts  of  the  range  under  consideration. 

The  value  of  this  integral  will  give  the  probability  that  a  randomly 
selected  value  of  the  variable  x  falls  within  the  range  a  ^  x  s  b. 

These  principles  can  be  applied  to  the  theory  of  reliability  to  obtain 
failure  density  and  distribution  functions.  Suppose  that  at  time  t  =  0, 
we  have  n  items  of  identical  age  and  failure  distribution  F(t).  As 
items  fail  they  are  not  replaced.  Therefore,  the  expected  number 
of  failures  by  some  time  t  is: 

n^tj)  =  nF(tj) 

It  follows  that  the  expected  number  of  items  surviving  to  a  time  t^  is 

n^tj)  =  n  -  nF(t1)=n[i  “  F^)] 

and  [1  —  F(t  )]  represents  the  probability  of  an  item  surviving  until 
time  t^.  This  is  also  referred  to  as  the  reliability  R(t^)  of  the  item 
for  time  t^. 

The  rate  at  which  items  fail  can  be  defined  as: 


_d 

dt 


[nF(t)l  =  nF'(t)  ~  nf(t) 
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I 


where  f(t)  represents  the  failure  density,  and  nf(t)  represents  the 
number  of  failures  occurring  during  the  next  increment  of  time 
following  time,  t. 


Another  important  relationship  used  in  reliability  measurement, 
the  hazard  rate  h(t),  is  obtained  as  the  ratio  of  the  failure  density 
function  to  the  survival  distribution  such  that 


h(t)  = 


nf(t) 


f(t) 


n[l~F(t)j  "  [1-  F(t)] 


This  function  gives  the  probability  of  failure  during  the  increment 
of  time  following  time  t,  assuming  the  item  has  survived  until  time 
t.  The  hazard  rate  is  a  function  of  the  time  already  operated,  and, 
in  general,  changes  with  time.  In  the  particular  case  of  the  expo¬ 
nential  distribution  of  failures,  however,  the  hazard  rate  remains 
constant  with  time,  and  is  identifiable  as  the  failure  rate  of  the 
item. 


In  summary,  four  basic  functions  are  used  in  reliability  measure¬ 
ment.  These  are: 


(a)  The  failure  density  function  f(t),  which  is  the  probability  that 
an  item  will  fail  at  a  given  time. 


(b)  The  failure  distribution  function,  given  by: 

F (t)  -  f(t)dt 


which  is  the  probability  that  the  item  will  fail  before  time  t. 


(c)  The  survival  distribution  function  or  reliability  function  is 
given  by: 


R 


:.(t)  =  1  -  F(t)  =  1  -  C  1  f(t)  dt  =  C°°  f(t) 


dt 


"t 


which  is  a  measure  of  the  probability  that  the  item  does  not 
fail  before  time  t. 


(d)  The  hazard  rate  function,  given  by: 

f(t) 


h(t)  = 


R(t) 


which  is  the  probability  that  the  item  will  fail  in  the  increment 
of  time  following  time  t,  given  that  it  survives  until  time  t. 
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In  practice,  specific  functions  must  be  substituted  for  the  general 
functions  in  the  above  expressions.  In  reliability  work,  the  most 
widely  used  functions  are  those  associated  with  the  exponential, 
Weibull  and  gamma  distributions.  These  distributions  are  de¬ 
scribed  briefly  below  in  relation  to  their  application  to  reliability 
measurement.  Other  distributions  are  also  used  in  reliability 
measurement.  However,  only  these  three  are  discussed  here 
as  being  representative  of  the  application  of  statistical  procedures 
to  reliability  problems. 

.  ?.  The  Exponential  Distribution. 

An  exponential  distribution  is  characterized  by  a  constant  instanta¬ 
neous  failure  rate  or  hazard  rate.  The  significance  of  this  is  that 
the  average  number  of  failures  which  occur  in  every  time  interval 
of  a  given  length  tends  towards  a  constant.  Physically,  a  constant 
failure  rate  indicates  that  the  items  have  gone  through  a  burn-in 
period  so  that  the  probability  of  failure  due  to  "bugs"  or  inherent 
design  deficiencies  is  reduced  to  a  negligible  quantity.  Also,  the 
system  has  not  reached  the  point  where  physical  wearout  of  com¬ 
ponents  may  result  in  an  increasing  failure  rate. 

The  failure  density  function  of  a  variable  which  has  an  exponential 
failure  distribution  is: 

f(t)  =  Xe  ** 

where  the  parameter  X  is  the  constant  failure  rat.'  and  -j-  or  0  is 

A. 

called  the  Mean-Time-Between-Failures  (MTBF).  The  failure 
density  function  has  the  value  X  at  (t-0)  and  declines  exponentially 
to  zero  as  time  approaches  infinity. 

The  reliability  function  is: 

R(t)  =  \  Xe  ^  dt  =  e  ^  for  t  >  0 
~t 

P  (t)  =  1  for  t  =  0 

Hence,  the  origin  of  the  term  exponential  failure  distribution.  The 
reliability  function  has  the  value  R(0)  =  1  and  goes  to  zero  asymptoti¬ 
cally  as  t  approaches  infinity. 

The  mean  of  the  exponential  distribution  is  (y)  and  the  variance  is  (-)  , 

The  failure  density  function,  reliability  function,  and  hazard  rate  ol 
a  typical  exponential  failure  distribution  are  shown  in  Figure  10-3. 


The  exponential  distribution  is  characterized  by  a  constant  failure 
rate  which  is  also  the  paraineter  of  the  distribution.  If  a  system 
has  survived  to  time  t,  the  probability  of  survival  for  the  next  time 
interval  of  length  t  is  the  same  as  if  it  had  just  been  placed  into 
service.  This  assumption  neglects  degradation  failures. 

The  exponential  distribution  is  the  most  commonly  used  distribution 
in  the  design  of  equipment  reliability  tests,  and  is  the  basis  of  the 
reliability  test  plans  presented  in  MIL,-STD-781B.  However,  care 
should  be  taken  in  applying  the  exponential  distribution  because  the 
failure  distribution  of  a  system  consisting  of  units  with  exponential 
failure  characteristics  is  not  always  exponential.  The  system  fail¬ 
ure  distribution  is  exponential  only  if  the  system  has  no  redundancy. 

2.  3  The  Weibull  Distribution. 

A  distribution  that  has  been  widely  used  in  describing  the  reliability 
characteristics  of  electronic  and  electromechanical  equipments  and 
parts  is  the  Weibull  distribution.  The  Weibull  distribution  is  capable 
of  describing  reliability  functions  characterized  by  failure  rates  that 
vary  with  time. 

The  Weibull  failure  density  function  is  defined  as: 

f(t;  a,  e,  y,  )  =  (0/a)  (t-  Y)8  1  exp  £  -  j  for  t  &  y 

Y  s  0 
0  >  0 
a  >  0 

=  0  elsewhere 

The  Weibull  failure  distribution  function  is: 

F (t)  =  1  -  exp  j"-  — - -  1 

L  a  J 

=  0  for  t  £  y 

The  three  parameters  in  the  equations  are: 

a  =  scale  parameter,  which  normalizes  the  function  with 
respect  to  the  variable. 

0  =  shape  parameter,  which  determines  the  general 

characteristics  of  the  function. 
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for  t  £  y 
a,  0  >  0 


V  -  location  parameter,  which  locates  the  "starting  point" 
of  the  function. 

The  shape  and  location  parameters,  pand  Y,  of  the  Weibull  distribu¬ 
tion  are  of  particular  interest  in  reliability  measurement. 

Values  of  g  greater  than  1  indicate  that  the  failure  rate  increases 
with  time,  while  values  of  0  less  than  1  indicate  a  decreasing  failure 
rate.  A  special  case  in  which  0  =  1  indicates  a  constant  failure  rate,  in 
which  case  the  Weibull  distribution  is  equivalent  to  the  exponential 
distribution.  For  example,  the  failure  distribution  function  where 
3  =  1  becomes: 

r  t  —  Y  ~ 

F(t)  =  1  -  exp  ^ - — 

This  indicates  a  distribution  in  which  no  failures  occur  until  after  a 
time  equal  to  Y.  If  Y  =  0,  failures  are  possible  at  anytime  after 
time  t  =  0.  The  failure  distribution,  when  3  =  1  and  Y  =  0  becomes: 


F  (t)  =  1  —  exp  £  —  ^ 


Letting  a  equal—  makes  this  exactly  equal  to  the  exponential  distribu- 
A. 

tion  as  discussed  in  paragraph  2.2. 

The  reliability  function  for  the  Weibull  distribution  is: 


R(t)  =  exp  — 


r  (t-YK 


t  s  Y 


R(t)  =  1, 


t  <  v 


Letting  0  =  1  and  Y  =  0  provides  the  commonly  used  exponential  re¬ 
liability  distribution. 

The  hazard  rate  function  for  the  Weibull  distribution  is: 


h  (t)  =  (0/a)  (t~Y) 


e-  i 


t  £  Y 


Letting  3  =  1  gives  h  (t)  =  — .  Thus,  the  parameter  —  becomes 

a  a 

equivalent  to  the  failure  rate  \of  the  exponential  distribution. 


i 


Figure  10-4  shows  a  graph  of  Weibuii  density  function,  reliability 
function,  and  hazard  rate  function.  Note,  that  as  B  increases  the 
positive  slope  of  the  nazard  rate  function  increases  rapidly.  This 
characteristic  of  the  hazard  rate  function  is  the  basis  on  which 
the  Weibull  distribution  is  frequently  selected  to  describe  the 
failure  characteristics  of  mechanical  parts  subject  to  wearout 
fa  i  1  u  r  e . 


Z.  4  The  Gamma  Distribution. 


S>-verai  typical  Gamma  distributions  are  shown  in  Figure  10-5. 

The  two  parameter  failure  density  functions  for  these  distributions 
given  by; 


f(t)  = 


l  ct  -  t/p  t  >  0 
a+  i  f  '• 


t<  0 


where  the  scale  parameter  8  >  0  and  the  shape  parameter  a  >  - 


The  failure  distribution  is  given  by: 


t  a  -x/3 


(  X  e 


F(l)  =  <1/a!b0  “f+t 


-  (1  /a!)  r[t/g(a+  i)  j 


where  r[t/6(0t  -l  1)]  is  the  incomplete  gamma  function  as  tabulated 
in  Karl  Pearson,  Tables  of  the  Incomplete  Gamma  Function, 
Cambridge  Univers ity  Press,  London,  1922. 


The  population  mean  and  variance  are  given  as  follows: 


U  =  3  (a  +  1 ) 


8  (a+  1) 
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Failure  Density  Functions,  Reliability,  and 
Hazard  Rate  for  the  Gamma  Distribution 


The  reliability  is  given  by: 


R 


<t)=  r - -ij  x“e  *x/Bdx=l~ir[t/e(a+l)],  t> 

a ! 0 


=  1 


t  =  0 


The  hazard  rate  is  obtained  from  the  expression: 


,  f(t)  —  t/e  /f»  a_-x/g  , 

h(t)  =  rTTTtc  /\tx 


The  failure  rate  asymptotically  approaches  a  constant  value  of  1/g 
with  increasing  time.  As  the  parameter  a  decreases  toward  zero 
the  failure  rate  at  a  given  time  increases  toward  1/p.  The  hazard 
rate  is  decreasing  for  a  <  0,  and  increasing  for  a  >0. 

The  reliability  function  of  the  Gamma  distribution  displays  an  in¬ 
teresting  behavior  pattern.  The  reliability  curve  tends  to  have  a 
flat  top  for  larger  values  of  a*  that  is,  the  reliability  maintains  at 
a  high  value  for  increasing  (t)  and  decreases  slowly.  As  a 
decreases  the  reliability  tends  to  decrease  towards  zero  more 
rapidly  and  at  any  given  time  the  reliability  is  lower  at  a  given 
time,  for  lower  a.  The  Gamma  distribution  becomes  the  exponen¬ 
tial  distribution  when  a  =  0. 


3.  BAYESIAN  STATISTICS  IN  RELIABILITY 


3.  1  Intr oducticn  to  Bayesian  Statistics, 


Bayesian  statistical  procedures  are  based  on  the  interpretation  of 
probability  as  a  degree  of  belief,  i.  e.  ,  the  probability  of  an 
event  A  is  a  measure  of  the  degree  of  belief  one  holds  in  the 
occurrence  of  the  event  A.  Under  this  interpretation,  probability 
may  be  directly  related  to  the  betting  odds  one  would  wager  on 
the  stated  proposition.  The  statement  that  the  probability  is 
0.  7  5  (or  equivalently,  that  the  odds  are  three  to  one)  that  a 
specified  change  in  the  design  will  improve  the  reliability  of  a 
given  product  involves  a  degree  of  belief  concerning  the  effect 
of  the  design  change  based  on  engineering  judgement  or  experience. 
This  differs  from  the  "usual"  frequency  definition  of  probab:.ity 
which  defines  probability  as  follows:  If  an  experiment  is  conducted 
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N  times  and  a  particular  event  A  occurs  n  times  tl  en  the  limit  of 
n/N  as  N  becomes  large  is  defined  as  the  probability  of  the  event  A. 
Note  in  the  design  change  example  no  experiment  has  been 
performed. 

Thus,  using  Bayesian  Techniques,  subjective  viev, ’points  (based 
on  experience,  guesses,  etc.  )  can  be  quantified  and  treated  under 
the  rules  of  probability.  Bayesian  procedures  rely  on  a  theorem 
by  Thomas  Bayes  called  Bayes  Theorem  which  allows  the 
combination  of  probabilities  (subjective)  and  experimental  data. 

This  theorem  can  be  stated  as  follows: 

{  The  probability  of  event  A.,  given  observed  data  } 


r 


=  prior  probability  of  A.  X 


Probability  of  Observed  \ 

V  Outcome  Under  Hypothesis  J 
Total  Probability  of  Observed 
Outcome  Under  All  Possible 
Hypotheses 


or  in  symbols : 


P(A.)P(B/A.) 

p(Ai  i  B»  ^ — 1 - ~ 

V  P(B/A.)P(A.) 

C-4  J  J 

j  =  l 


where  B  is  the  observed  outcome  and  A.  is  the  specific  event 
whose  probabiiity  is  being  calculated.  ¥he  discrete  probabilities 
above  can  be  replaced  by  continuous  functions.  The  left  hand 
side  of  the  equation  is  often  called  the  posterior  probability  of  A.. 

3.2  Use  of  Bayesian  Techniques  in  Equipment  Reliability  Demonstration. 

Bayesian  procedures  lend  themselves  for  use  during  those  stages 
of  reliability  programs  when  little  test  data  are  available,  and 
when  it  would  be  desirable  to  combine  "engineering  judgement 
and  experience"  with  the  available  test  data  to  arrive  at  an 
estimate  of  the  reliability  of  the  equipment.  The  use  of  these 
procedures  is  described  below.  The  purpose  of  the  discussion 
is  to  illustrate  the  technique  and  not  to  give  a  detailed  description 
of  the  procedure. 
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Assume  that  an  engineer's  prior  estimate  (based  on  inherent 
failure  rate  estimation  or  reliability  prediction  etc.)  of  the  re¬ 
liability  of  an  equipment  is  .  90.  This  information  is  encoded 
into  a  Bayesian  framework  by  supplying  a  measure  of  the  degree 
of  belief  that  the  reliability  is  .90.  Techniques  for  doing  this 
are  omitted  in  this  introduction.  However,  for  the  case  where 
the  test  data  are  in  the  form  of  the  number  of  equipments  pass¬ 
ing  a  given  test,  the  prior  information  is  encoded  by  specifying 
a  number  of  assumed  test  trials  and  assumed  test  successes. 

If  an  engineer  believed  strongly  in  the  prior  reliability  estimated 
he  would  use  a  large  number  of  assumed  test  trials.  If  he  did 
not  have  great  faith  in  the  prior  estimate  (diffuse  prior)  he  would 
use  a  smaller  number  of  assumed  test  trials.  The  number  of  as¬ 
sumed  successes  would  be  chosen  so  that  the  ratio  of  assumed 
successes  to  assumed  test  trials  was  equal  to  the  prior  estimate. 


Hence,  for  encoding  the  .  90  reliability  above  v/ith  a  very  strong 
belief  one  could  use: 


Prior  estimate  of  reliability  = 


900  assumed  successes 
1000  assumed  trials 


=  .  90. 


For  a  prior  estimate  whose  validity  was  not  as  strongly  believed 
one  could  use: 


Prior  estimate  of  reliability 


90  assumed  successes 
100  assumed  trials 


90 


A  still  "weaker"  prior  would  be  encoded: 


Prior  estimate  of  reliability 


9  assumed  successes 
10  assumed  trials 


90 


The  effect  of  combining  test  data  with  these  prior  estimates  can 
be  illustrated  by  assuming  that.  19  out  of  20  equipments  pass  a 
reliability  test.  The  Bayesian  expression  for  this  case  would  be: 


Posterior  estimate  of  reliability  = 

number  assumed  successes  +  number  observed  successes 
number  assumed  trials  +  number  observed  trials 
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For  the  three  above  cases  this  yields: 


900  +  19  p  90+19 

1000  +  20  '  *  '  ’  2  ~  100  +  20 


=  .  909,  and  P3 


9+19 
10  +  20 


=  .  933 


The  strongly  believed  priors  were  little  affected  by  the  test  data 
while  the  weaker  prior  was  changed.  In  general,  the  weaker  the 
prior  the  less  is  its  effect  on  the  estimate  for  a  given  set  of  data. 
Notice  in  this  case,  P  .is  close  to  the  estimate  of  0.  95  using  the 
data  alone. 


This  characteristic  is  even  more  apparent  when  the  test  results 
are  far  removed  from  the  prior.  For  example,  if  there  had  been 
no  observed  successes  in  the  above  illustration,  i.  e.  ,  if  20  suc¬ 
cessive  failures  are  observed,  then: 


P  ..  900  _  p  _  _90 

P1  1020  "  ‘  '  P2  120 


=  .  75,  and  P. 


q 

30 


.  30 


The  strong  prior  has  been  little  changed  by  the  data  and  a  posterior 
estimate  of  .  882  which  is  close  to  .  90  is  obtained  in  spite  of  20 
successive  failures.  In  this  case,  is  more  strongly  influenced 
by  the  estimate  of  zero  successes  based  on  the  data  alone. 

Thus  strong  (non-diffus e)  priors  should  not  be  used.  In  fact,  if  a 
strong  prior  is  used  there  is  no  need  for  a  test  program  since  'he 
test  results  are,  in  fact,  ignored.  On  the  other  hand,  diffuse 
priors  "disappear"  quickly  in  the  presence  of  test  data  and  esti¬ 
mates  that  agree  closely  with  those  based  on  the  data  itself.  Weak 
priors,  however,  have  certain  advantages  since  they  provide  some 
basis  of  assessing  reliability  early  in  the  program  when  little  test 
data  is  available;  they  can  be  combined  with  observed  results,  thus 
avoiding  estimates  of  0  or  1  which  are  intuitively  non- appealing 
(using  data  alone  5  out  of  5  successes  would  yield  a  reliability  esti¬ 
mate  of  1.0),  and  thus  allow  the  sequential  combination  of  test 
results . 


Let  us  consider  another  example  using  continuous  distributions. 

An  equipment  is  tested  until  10  failures  are  observed.  The  tenth 
failure  occurred  after  1000  operating  hours  had  been  logged.  We 
wish  to  obtain  an  estimate  and  a  lower  95  percent  bound  on  the 
mean  life  of  the  equipment  using  a  prior  distribution  (degree  of 
belief)  of  the  parameter  of  the  equipment  failure  distribution  com¬ 
bined  with  the  test  data.  The  equipment  MTBF  requirement  is  90 
hours  with  a  lower  95  percent  bound  on  the  mean  life  to  be  at  least 
70  hours . 


10-17 


The  posterior  probability  of  A.  as  shown  before  is  given  by 


P(A.IB)  =  - 
i  1  n 


P(A.)  P(B  A.) 
1  1 


y  P(B  A.)  P(A.) 

J  J 

j  =  l 


and  for  continuous  functions  it  is 


P(A.  |  B) 


f(X)  P(t|\) 


\  f(X)  P(t|  X)  d  A 
J0 


where: 


f(X)  is  the  density  function  of  the  prior  distribution 

P(tjx)  is  the  density  function  of  the  observed  outcome 
given  X 

\  f (X )  P(t|  X)  d\  is  the  integral  of  the  conditional  densities 

0  weighted  according  to  the  densities  of  the  respec¬ 
tive  prior.  (Joint  probability  distribution) 

X  is  the  parameter  of  the  equipment  failure  distribution. 


Assume  that  the  equipment  on  test  fails  exponentially;  that  is, 
the  failure  density  function  f(t)  is  given  by 


f(t)  =  Xe 


-  Xt 


where: 

X  is  the  constant  failure  rate 

and  our  degree  of  belief  that  X  is  of  different  magnitudes  (the  prior 
density)  is  given  by  a  gamma  density  function  f(X)  with  parameters 
0  =2.0  and  |3  =  .  0025, 


The  mean  (la  )  of  the  prior  is 
A. 

H  =  3(o  +  1)  =  .  0025(3)  =  .  0075 

A 
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and  the  variance  (a  )  of  the  prior  is 


cr  2  =  92 (a  +  1)  =  .00000625(3)  =  .00001875 


The  density  function  of  the  test  data  given  X  is 
k  . 

Ti  -  At  k  -  XT  10  _  1000X 

P(t  I X)  =  11  (ke  )  =  X  e  -  X  e 

i  =  l 


where: 


X  is  the  parameter  -  a  constant  failure  rate 

k  is  the  number  of  failures  =  10 
th. 

t  is  the  i  time-between  failures 
l 

T|  is  the  sum  of  the  k  times  between  failures  =  1000  hours 


and  the  ^  f(X)  P(t  |  X)  d  X 


■  f 


1  X%-X/eXke"XTk 


'0  a!  (3 


a  +  1 


a  +  k  -  x(-jr  +  ^ 


a !  B 


Vtt  \ 


X  e 


dX 


0 


Making  the  transformation  |j  =  X  ( —  + 


with  dX  =  — — - djj,  X  =  ” — — 


—  +  T 
8  k 


+  T. 

P  k 


we  obtain 


a !  B 


_1_  _  CcV  M  X°+k  U 

,o+l  ill 


0  y  +  Tk 


—  +  T 

P  k 


dd 


a!  B 


1  /  1  ,°  +  k  +  1  r. 

o+l  1 


B  +  Tk 


a+k  —  u 

U  e  dia 
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(o  +  k)! 


a!B°  +1(t  +  Tk)a+k+1 


Thus  the  posterior  density  function  P(A.|B)  is  given  by: 


P(A.|B) 


1  ,  a  —  X.  /  0  ,  k  -  *Tk 

- r  A,  e  X  e 

•, !  _ 

_ •  (g  +  k)! _ 

.  .  a  +  1  /  1  „ ,  \  a  +  k  +  1 

a!B  (t+  \) 


(C4  k)! 


a+k  -  x(e  +  Tk) 
X  e 


which  is  a  gamma  density  function  with  parameters 

fl1  =  - - -  =  — 7 - - -  -  — 1-~  =  0.  0007142857 

1  _  I  ,  1400 

T  +  Tk  .0025  +  1000 


a  =  a  +  k  =  2.  +10.  =  12. 


The  mean  (u,  )  of  the  posterior  is 

A, 

p  =  p1  fa1  +  i  )  =  0.  00°7142857(13)  -  0. 

and  the  variance  is 


0092857142 


2  =  p1  (a1  +  l)  =  0.  000000510204(13)  =  0.  00000663265 


The  mean  (p  )  is  the  mean  failure  rate,  the  parameter  of  the  expo- 
X 

nential  equipment  failure  distribution;  hence  its  reciprocal  is  the 
mean  life  (p)  of  the  equipment. 


U  =  —  =  107.7  hours 

^X 
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A  lower  95  percent  bound  on  the  mean  life  (|i)  is  equal  to  the 
reciprocal  of  an  upper  95  percent  bound  on  the  mean  failure  rate  (u  ^  ). 
The  upper  95  percent  bound  on  the  mean  failure  rate  (U^)  is  found 
by  solving  the  following  equation  for  u  ^ 


M'  (tV) 

—  -FT 

e  k 


a  +  k  +  1 


,  a+  k 
X  e 


dX  =  .  95 


\  XUB 


.12  -  X/0. 0007142857,, 

X  e  dX  =  .  95 


12!  (0. 0007142857) 


0. 0142295 


Thus  the  lower  95  percent  bound  of  the  mean  life  (p  )  is 

JljJd 


lb  -  (i 


=  70. 3  hours 


The  'Bayesian  analysis  indicates  that  the  equipment  meets  the 
MTBF  requirement  of  90  hours  with  a  lower  95  percent  bound  on 
the  mean  life  of  at  least  70  hours. 

The  advantages  of  the  Bayesian  technique  is  that  the  test  time  and 
cost  of  an  equipment  reliability  demonstration  test  can  be  reduced 
using  the  right  prior.  Investigations  are  presently  being  performed 
for  determining  typical  priors  for  use  in  determining  realistic 
posteriori  distributions.  For  example  the  use  of  conjugate  dis¬ 
tributions  as  priors  (i.  e.  ,  the  gamma  distribution  with  the  expo¬ 
nential,  the  beta  distribution  with  the  binomial,  etc,  )  are  being 
investigated  for  equipment  reliability  demonstration  tests. 


The  disadvantage  of  the  Bayesian  technique  is  that  the  use  of  the 
wrong  prior  can  lead  to  the  wrong  decision  or  an  extremely  lengthy 
test  time  where  estimates  are  initially  erroneous  and  which  may 
not  disappear  until  a  substantial  amount  of  test  results  are  ac¬ 
cumulated.  This  might  not  occur  until  late  in  the  program  and 
require  major  modifications  whose  need  might  have  been  apparent 
early  had  not  Bayesian  techniques  been  used.  Thus,  great  care 


10-21 


must  be  taken  in  determining  the  form  and  parameters  of  the 
prior  distribution  (method  of  encoding  information).  Frequently, 
this  is  a  difficult  problem  in  that  the  subjective  prior  distribution 
may  depend  heavily  on  the  past  experience  and  prejudices  of  the 
person  making  the  estimate.  It  has  been  suggested  that  a  standard 
set  of  questions  be  established  which  would  aid  workers  in  defining 
the  characteristics  of  the  prior  distribution.  For  example,  the 
analyst  might  be  requested  to  define  the  0.  01 ,  0.25,  0.5,  0.75, 
and  0.  99  points  of  the  subjective  distribution.  From  this  infor¬ 
mation  the  form  and  parameters  of  the  prior  distribution  can  be 
deduced. 

3.3  Concluding  Remarks. 

The  use  of  Bayesian  procedures  in  reliability  estimation  tech¬ 
niques  is  new  and  controversial.  Many  statisticians  do  not  accept 
as  meaningful  the  subjective  interpretation  of  probability  which 
the  Bayesian  techniques  utilize  and  hence  argue  the  position  that 
the  probability  statements  are  meaningless. 

The  above  material  has  attempted  to  describe  the  procedures  and 
how  they  might  be  used.  Some  of  the  advantages  and  disadvantages 
have  been  delineated.  If  one  wishes  to  use  such  procedures  ,  cau¬ 
tion  should  be  used  and  results  should  be  closely  monitored.  In 
any  case  a  more  thorough  technical  description  than  is  presented 
here  should  be  consulted. 

RELIABILITY  TESTING 

Reliability  testing  involves  an  empirical  measurement  of  times -to- 
failure  during  equipment  operation  for  the  purpose  of  determining 
whether  an  equipment  meets  the  established  reliability  requirements. 

A  reliability  test  is  effectively  a  "sampling"  test  in  the  same  sense 
that  it  is  a  test  involving  a  sample  of  objects  selected  from  a  "popula¬ 
tion".  In  reliability  testing,  the  "population"  being  measured  encom¬ 
passes  all  failures  that  will  occur  during  the  life  span  of  the  equipment. 
A  "test  sample"  is  drawn  from  this  population  by  observing  those  fail¬ 
ures  occurring  during  a  small  portiop  of  the  equipment's  life. 

In  reliability  testing,  as  in  any  sampling  test,  the  "sample"  is 
assumed  to  be  representative  of  the  population,  and  the  mean  value 
of  the  various  elements  of  the  sample  (times -to-failure)  is  assumed 
to  be  a  measure  of  the  true  mean  (MTBF,  etc.  )  of  the  population.  It 
is  recognized,  however,  that  a  certain  amount  of  variability  exists  in 
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the  population,  and  that  it  is  unlikely  for  a  randomly  selected  sample 
to  have  exactly  the  same  mean  as  the  population  or  a  mean  value  quite 
different  from  the  population  mean. 

Sampling  errors  in  a  reliability  test  cannot  be  avoided.  However,  the 
application  of  statistical  techniques  permits  determining  the  probable 
magnitude  of  sampling  error.  Thus,  the  level  of  confidence  that  can 
be  placed  in  the  results  of  a  reliability  test  can  be  quantified. 

The  following  discussion  illustrates,  in  a  simplified  manner,  certain 
basic  relationships  between  the  .various  parameters  of  a  reliability 
test. 

4.1  Mechanics  of  a  Reliability  Test. 

In  any  sampling  test,  conclusions  are  drawn  based  on  the.  charac¬ 
teristics  of  a  test  "sample"  which  is  selected  at  random  from  a 
"population".  A  sample  in  a  reliability  test  consists  of  a  number 
of  times -to-failure,  and  the  population  is  all  the  times -to-failure 
that  could  occur  either  from  the  one  equipment  or  the  more  than 
one  equipment  on  test.  The  test  equipments  (assuming  more  than 
one)  are  considered  identical  and  thus  their  populations  are  also 
identical.  Therefore  a  random  sampling  from  one  population  or 
from  all  populations  will  produce  the  same  statistical  results. 

The  sample  is  characterized  by  the  mean  value  (0  )  of  all  times- 

1  b 

to-failure  in  the  sample. 

If  all  possible  samples  of  the  same  number  of  times -to-failure 
were  drawn  from  the  same  or  identical  equipment,  the  resulting 
set  of  sample  means  would  be  distributed  is  some  manner,  as 
illustrated  in  Figure  10-6.  The  true  MTBF  (0)  of  the  equipment 
is  not  known,  but  sample  means  will  be  distributed  around  0  in 
some  predictable  manner. 

Assume  that  one  sample,  having  a  mean  8  is  drawn  from  the 

o 

distribution  illustrated  in  Figure  10-6.  This  one  sample  mean  is 
to  be  used  to  determine  whether  the  population  mean  9  can  be 
expected  to  be  greater  than  some  previously  specified  minimum 


i 

*  Other  parameters,  such  as  standard  deviation,  or  variance  are 
also  used  to  describe  a  sample.  However,  only  the  mean  is  used 
here  for  convenience,  and  to  be  compatible  with  MIL-STD- 781B, 
which  considers  only  the  one-parameter  exponential  distribution 
of  failures. 


acceptable  value,  0 


^ .  If  the  true  mean  were  exactly  equal  to  6^ , 
then  0  could  be  substituted  lor  8  in  Figure  10-6.  Also,  0  could 

1  O 

now  be  located  at  some  point  on  the  abscissa  as  shown  in  Figure 
10-7. 


The  shaded  area  in  Figure  10-7  represents  the  percentage  of  all 
samples  that  would  have  had  a  mean  equal  to  or  less  than  0  when 

the  true  mean  is  equal  to  0  .  Also,  the  area  marked  0  repre- 

1  o 

sents  the  percentage  of  samples  that  would  have  had  a  mean  greater 
than  0g.  It  could,  therefore,  be  concluded  that  there  is  a  prob¬ 
ability  of  R0  percent  that  a  sample  selected  at  random  from  a  pop- 
o 

ulation  having  a  true  mean  equal  to  9^  will  have  a  sample  mean 

higher  than  0^.  A  more  meaningful  interpretation  would  be  that 

after  obtaining  a  sample  mean  of  0  ,  there  is  a  0  percent  chance 

o  o 

that  the  equipment  tested  had  a  true  MTBF  as  low  as  0^ .  The 

equipment  could  be  considered  to  be  within  specifications  providing 
f3g  does  not  exceed  the  pre-specified  acceptable  level  of  consumer's 

risk. 


4.  2  Effect  of  Sample  Size  on  Reliability  Testing. 

In  the  above  discussion,  the  shape  of  the  distribution  curve  was 
considered  to  be  related  to  the  selection  of  samples,  each  con¬ 
sisting  of  a  given  number  of  failures.  Changing  the  sample  size, 
however,  will  result  in  a  change  in  the  shape  of  the  distribution 
curve.  For  example,  if  the  sample  size  is  increased,  the  sample 
means  will  tend  to  cluster  more  closely  around  the  true  MTBF, 
and  the  chance  of  sampling  error  will  be  reduced.  The  curves  in 
Figure  10-8  illustrate  two  different  distributions  of  sample  means 
that  could  be  drawn  from  the  same  equipment.  The  narrower 
curve  represents  the  distribution  of  sample  means  when  samples 
of  larger  size  are  drawn.  The  shaded  areas  under  each  curve 
represent  the  respective  proportion  of  times  the  sampling  error 

would  be  equal  to  8  —  8  or  greater.  The  larger  sample  size 

o  1 

would  be  much  less  likely  to  be  in  error  by  any  given  amount. 
Thus,  greater  precision  in  reliability  testing  will  be  realized  as 
the  number  of  time3 -to-failure  observed  during  the  test  increases. 
Unfortunately,  this  greater  precision  can  only  be  obtained  at  the 
expense  of  testing  time  and  cost.  Therefore,  a  trade-off  area 
exists  between  testing  precision  and  economy  of  testing. 
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Sample  Means 
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4.3  Minimum  Acceptable  MTBF  (6^),  Accept/Reject  Criterion,  and 
Consumer's  Risk  (R)  Relationships. 

Knowledge  of  the  relationships  between  sample  size  and  tne  ex¬ 
pected  distribution  of  sample  means  permits  the  development  of 
test  plans  which  include  a  pre-determined  accept/reject  criterion 
as  a  part  of  the  plan.  This  criterion  is  effectively  the  value  of  a 
sample  mean  that  should  be  obtained  no  more  than  some  small 
percent  of  the  time  when  a  sample  is  drawn  from  an  equipment 
having  a  true  M7  BF  equal  the  minimum  acceptable  value  8  . 

For  example,  if  the  value  Q  in  Figure  10-7  represents  the  maxi- 

mum  level  of  risk  that  is  to  be  assumed  in  accepting  an  equipment 

having  a  MTBF  as  low  as  0  ,  then  the  value  8  would  be  established 

1  o 

as  the  accept/ reject  criterion.  If  a  sample  mean  obtained  as  a 
result  of  a  reliability  test  were  equal  to  or  greater  than  this  value, 
then  the  risk  of  accepting  an  equipment  having  a  MTBF  less  than 
0  would  be  less  than  R  ,  and  the  equipment  could  be  accepted. 

For  convenience,  accept/reject  criteria  are  usually  stated  either 
in  terms  of  the  number  of  failures  occurring  during  a  given  oper¬ 
ating  time  interval,  or  the  time  elapsing  before  a  given  number  of 
failures  occur.  In  either  case,  the  number  of  failures  and  time 
interval  combinations  are  equivalent  to  the  0g  value  as  discussed 

above.  The  accept/ reject  criteria  are  directly  related  tc  the 
minimum  acceptable  MTBF  (8^),  the  acceptable  level  of  "con¬ 
sumer's  risk,  "  (R),  and  the  sample  size. 

However,  knowledge  of  the  variance  of  the  time-to-failure  distri¬ 
bution  is  not  usually  known  prior  to  test.  One  option  is  to  utilize 
the  exponential  distribution  assumption  of  the  variance  equal  to  the 
mean.  This  assumption  will  provide  conservative  results  if  the 
actual  time-to-failure  distribution  has  an  increasing  rather  than 
constant  failure  rate.  Another  option  is  to  use  the  test  data  to 
provide  an  estimate  of  the  mean  and  variance,  in  which  case  the 
accept/reject  criterion  cannot  be  determined  in  advance  of  the 
test, 

4.4  Producer's  Risk  (a)  and  Producer's  MTBF  Goal  (0  ) 

G 

Consideration  of  the  relationships  between  minimum  acceptable 
MTBF  (0j),  consumer's  risk  (0),  and  sample  size,  as  discussed 

above,  will  permit  the  customer  (Air  Force)  to  control  the  risk 
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being  taken  in  testing  to  assure  that  the  product  is  acceptable. 
However,  there  is  also  the  risk  to  the  producer  of  having  a  truly 
acceptable  equipment  rejected. 

It  is  possible  that  sampling  error  will  produce  a  sample  having  a 
mean  less  than  the  accept/reject  value  and,  therefore,  cause  re¬ 
jection,  even  though  the  true  MTBF  of  the  equipment  is  greater 
than  the  minimum  acceptable  value.  This  possibility  is  illustrated 
in  Figure  10-9.  The  left-hand  curve  in  this  figure  represents  the 
distribution  of  sample  means  when  the  true  MTBF  is  just  equal  to 
the  accept/reject  value.  In  this  case,  there  is  a  50  percent  chance 
of  having  the  product  rejected  due  to  sampling  error.  This  chance, 
or  risk  of  rejecting  an  acceptable  equipment, is  represented  by  the 
shaded  area  designated  a  The  risk  of  rejecting  a  "good"  equip¬ 
ment  will  be  less,  however,  if  the  equipment  MTBF  is  greater 
than  the  accept/ reject  value.  The  shaded  area  designated  a.^  in 

Figure  10-9  represents  the  probability  of  rejecting  an  equipment 
having  a  true  MTBF  greater  than  the  accept/reject  value.  It  is' 
apparent  that  this  risk  will  be  reduced  as  the  true  MTBF  and/or 
the  sample  size  is  increased. 

To  assure  the  acceptance  of  his  equipment,  the  contractor  estab¬ 
lishes  a  reliability  goal  that  is  somewhat  higher  than  the  accept/ 
reject  value.  This  MTBF  goal  (0^)  is  selected  such  that,  if  the 

equipment  MTBF  is  truly  equal  to  0„,  the  risk  of  the  equipment 

G 

being  rejected  due  to  sampling  error  will  be  not  greater  than  some 
pre-established  producer's  risk  \a). 

This  risk  (a)  can  be  reduced,  either  by  increasing  0  with  respect 

u 

to  the  accept/ reject  value,  or  by  increasing  the  sample  size.  How¬ 
ever,  the  latter  alternative  also  affects  the  relationship  between  the 
accept/ reject  criterion  and  the  minimum  acceptable  MTBF  (0^),  as 

discussed  in  paragraph  4.  3.  In  determining  a  test  plan,  the  two 
risks  (a  and  0)  are  usually  pre-established  to  be  equal. 

4.5  Combined  Characteristics  of  a  Reliability  Test  Plan. 

In  practice,  a  reliability  test  must  satisfy  two  objectives.  First, 
it  must  assure  the  government  that  an  unacceptable  equipment  is 
rejected.  However,  the  contractor  must  also  be  assured  that  an 
acceptable  equipment  is  not  rejected.  Since  100  percent  assurance 
of  meeting  either  obj*.  ^  tivre  is  net  possible  in  any  practical  case, 
both  the  government  a.  1  th  contractor  must  be  willing  to  assume 
some  degree  of  risk  in  uesigning  an  '  performing  the  test. 
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Figure  10-8.  Distribution  of  Sample  Means  for  Large  and 
Small  Sample  Sizes. 


Figure  10  -9.  D.istr  ibutions  of  Sample  Means  Drawn 
From  Truly  Acceptable  Equipments 


The  Government  establishes  a  minimum  acceptable  MTBF  (0  ^ ) 
and  accepts  some  risk  (g)  in  verifying  whether  the  achieved  MTBF 
is  truly  greater  than  (0  ).  On  the  other  hand,  the  Government  has 
an  MTBF  requirement  (8q)»  but  because  of  test  time,  schedule, 
dollars,  etc.  limitations,  the  Government  is  willing  to  accept  a 
demonstrated  gj.  It  is  with  respect  to  the  MTBF  requirement  that 
the  Government  specifies  a  contractor  risk  (a)  of  having  his  equip¬ 
ment  rejected,  even  though  the  design  requirement,  8q,  isachieved. 
The  two  values,  0^  and  0^,  and  the  cor  responding  risks,  0  and  a, 
are  directly  related  because  the  results  of  a  reliability  test  willbe 
evaluated  with  respect  to  only  one  accept/ reject  criterion  as  shown 
in  the  test  plans  in  MIL-STD-781  B .  This  relationship  is  primarily 
a  function  of  the  distribution  of  sample  means  and,  thus,  is  directly 
influenced  by  the  number  of  failures  and  variances  of  the  times-*o- 
failure  obtained  in  the  test.  Of  course,  the  contractor  sets  his  own 
design  goal  9  which  is  usually  around  9  according  to  a  cost/risk/ 
profit  evaluation  he  makes. 

The  basic  relationships  to  be  considered  in  test  plan  design  can  be 
illustrated  by  considering  two  hypothetical  equipments,  one  having 
a  MTBF  at  exactly  the  minim’im  acceptable  value,  0j,  and  the  other 
exactly  meeting  the  design  requirement,  6q,  which  is  greater  than 9j. 
Also,  the  true  MTBF  of  either  equipment  is  not  known,  but  a  reli¬ 
ability  test  is  to  be  performed  to  distinguish  between  the  acceptable 
and  unacceptable  equipment.  The  problem  is  to  select  a  sample  size 
such  that  the  test  will  discriminate  between  the  two  equipments  with 
only  an  acceptable  level  of  risk  of  milking  incorrect  decisions  due 
to  sampling  error. 

By  selecting  a  sufficiently  large  sample  size,  the  distribution  of 
sample  means  from  the  two  equipments  could  be  made  to  appear  as 
illustrated  in  Figure  10-10  (a).  In  this  case,  there  would  be  almost 
no  ambiguity  concerning  the  particular  equipment  from  which  the 
sample  was  drawn.  This  plan  wotald  readily  discriminate  between 
the  two  equipments, with  negligible  risk  of  incorrect  decision  due  to 
sampling  error. 

A  test  plan  that  provides  this  degree  of  discrimination  is  usually 
impractical  because  it  usually  requires  a  prohibitively  large  sample 
size  for  the  ratio  of  8q  to  that  was  established.  Therefore,  in 
most  reliability  tests,  the  sample  size  is  usually  set  as  low  as  pos¬ 
sible  to  reduce  testing  cost  and  time  requirements  by  specifying 
the  maximum  acceptable  a  and  9  risks  that  can  be  associated  with 
0q  and  the  smallest  acceptable  0^  respectively. 
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[f] 


(a)  Large  Sample  Size  Distributions 


Figure  10-10.  Characteristics  of  Reliability  Test  Plans 
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Tile  effect  of  reducing  the  sample  size  is  illustrated  in  Figure 
10-10  (b).  Here,  0.,  0-.  and  the  accept/reject  value  are  the  same 
as  in  Figure  1  0-1  0(a),  but  the  sample  size  has  beer,  reduced, 
thereby  broadening  the  distribution  curves  as  shown.  The  areas 
designated  a  and  0  represent  the  resulting  producer's  and  con¬ 
sumer's  risks,  respectively.  The  effect  of  a  significant  decrease 
in  0  will  be  a  significant  decrease  in  the  0  risk  and  a  decrease 
in  the  a  risk  whereas,  the  effect  of  an  increase  in  0  j  will  be  the 
opposite.  The  effect  of  a  significant  increase  in  0q  will  be  a 
significant  decrease  in  the  a  risk  and  a  decrease  in  the  0  risk, 
whereas  the  effect  of  a  decrease  in  0q  will  be  the  opposite. 

As  indicated  by  the  preceding  discussion,  the  risks  involved  in  a 
reliability  test  are  affected  by  sample  size,  the  relationship  be¬ 
tween  0^  and  8q  ,  and  the  accept/ re  jec.t  criterion.  Therefore,  the 
design  of  a  reliability  test  involves  the  consideration  of  the  rela¬ 
tionship  between  the  various  parameters  and,  to  the  extent  possible, 
to  minimize  the  risk,  cost  and  time  requirement. 

In  practice,  the  sample  size  is  often  specified  in  terms  of  a  fixed 
number  of  equipment  failures.  Alternate’ v,  however,  the  test 
time  can  be  fixed,  and  the  sample  size  will  be  the  number  of 
failures  occurring  during  this  time  interval. 

The  relationship  between  and  0q  is  usually  stated  in  terms  of 
the  "discrimination  ratio",  6q/0j.  This  parameter  is  applicable 
when  the  individual  times -to- failure  of  the  equipment  are  exponen¬ 
tially  distributed.  Other  distributions  will  require  the  considera¬ 
tion  of  other  parameters,  such  as  the  variance  or  standard  devia¬ 
tion  in  relating  0^  to  0^  . 

The  accept/reject  criterion  is  usually  sta  ed  in  terms  of  a  specific 
number  of  failures  occurring  during  the  duration  of  the  test.  Some 
accept/reject  criteria  are  stated  in  terms  cf  a  maximum  allowable 
number  of  failures  during  a  fixed  time  interval,  while  others  are 
stated  in  terms  of  a  minimum  time  permitted  before  a  fixed  number 
of  failures  occur.  In  either  case,  the  criterion  corresponds  a 
particular  MTBF  value  falling  between  and  0  . 

In  most  Air  Force  procurement  programs,  reliability  test  plans 
are  designed  using  MIL-STD-781 B,  Reliability  Tests,  Exponential 
Distribution.  This  standard  provides  data  suitable  for  designing 
test  plans  when  the  exponential  distribution  of  failures  can  be  as¬ 
sumed.  Other  sources  of  sampling  plan  data  are  available  for  ap¬ 
plication  when  distributions  other  than  exponential  are  expected. 

Some  of  these  sources  are  mentioned  in  paragraph  8  of  this  chapter. 
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Some  considerations  in  the  application  of  MIL-STD-781B  in  the 
design  of  reliability  test  plans  are  presented  in  the  following 
paragraphs. 


RELIABILITY  TEST  PLAN  DESIGN 

The  Air  Force  is  responsible  for  establishing  the  initial  reliability 
requirement,  and  the  desired  degree  of  assurance  that  this  require¬ 
ment  is  met.  The  contractor  must  then  establish  his  design  goals 
and  develop  a  test  program  that  verifies  achievement  of  these  goals 
to  the  satisfaction  of  the  Air  Force,  and  at  acceptable  risk  and  min¬ 
imum  cost  to  the  contractor.  All  of  these  factors  are  interrelated 
and  must  be  considered  in  the  design  of  a  test  plan.  In  general,  a 
reliability  test  plan  is  defined  by  establishing  the  minimum  accept¬ 
able  MTBF  (6^),  the  consumer's  risk  (0),  and  the  specified  MTBF 

(0q),  the  producer's  risk  (a),  and  the  accept /reject  criterion.  The 

latter  parameter  relates  total  test  time  to  observed  failures  and, 
therefore,  effectively  establishes  the  sample  size. 

In  general,  reliability  tests  performed  under  Air  Force  contract  are 
designed  in  accordance  with  the  provisions  of  MIL -STD- 78 IB, 
"Reliability  Tests:  Exponential  Distribution".  This  standard  pro¬ 
vides  a  series  of  test  plans,  test  levels,  and  procedures  applicable 
to  lecti  onic  equipment  reliability  testing  where  the  exponential 
failure  distribution  can  be  assumed.  The  application  of  this  stan¬ 
dard  involves  the  performance  of  a  series  of  tasks  in  establishing 
the  test  plan  parameters.  The  following  paragraphs  contain  data 
and  discussions  to  aid  in  the  application  of  MIL-STD-781B. 

5.  1  Establishing  the  Minimum  Acceptable  MTBF  (0^). 

The  minimum  acceptable  MTBF  (8^)  is  a  value  determined  by 
the  Air  Force  to  be  the  minimum  level  of  MTBF  that  can  be 
tolerated.  Any  equipments  having  a  MTBF  less  than  0^  are 

considered  unsatisfactory.  This  value  should  be  carefully 
selected,  and  should  reflect  the  reliability  allocations  per¬ 
formed  in  developing  the  System  Specification.  Also,  con¬ 
sideration  should  be  given  to  the  risk  of  inadvertently  accept¬ 
ing  an  unacceptable  equipment  due  to  the  inherent  variability 
of  a  reliability  test.  It  should  be  noted  that  reliability  test 
plans  with  consumer's  risks  (P)  of  less  than  10%  of  accepting 
a  true  MTBF  equal  to  0^  are  usually  not  practical.  Therefore, 

if  any  uncertainly  exists,  8^  values  are  usually  selected  in  a 
somewhat  conservative  manner. 
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5.2  Establishing  the  Consumer's  Decision  Rusk  (P). 


The  degree  of  risk  that  the  Air  rm-ce  is  willing  to  assume  in 
accepting  an  unacceptable  equipment  should  be  established  at 
the  time  6^  is  established.  This  risk,  which  is  measured  in 

terms  of  the  probability  (f3)  of  accepting  an  equipment  having 
a  true  MT  BF  equal  to  6^  is  chosen  depending  on  the  im¬ 
portance  of  achieving  the  specified  level  of  reliability,  and  on 
the  practicability  of  test  plans  that  would  provide  given  risk 
levels . 


MIL-STD-781B  provides  reliability  test  plans  having  P's 
ranging  from  10%  to  40%.  (Test  plans  having  p's  as  low  as 
1%  are  available  in  other  sources  such  as  Handbook  H108.^ 
However,  economic  and  time  limitation  factors  usually  pro¬ 
hibit  the  use  of  the  iower  risl  levels.  )  A  point  of  departure 
for  establishing  a  P  value  can  be  obtained  by  considering  the 
general  relationship  between  this  parameter  and  the  subse¬ 
quent  reliability  test  time  requirements  of  MIL-STD-781B 
test  plans.  For  example,  the  approximate  ranges  of  ex¬ 
pected  test  time  for  commonly  used  test  plans  having  a  dis¬ 
crimination  ratio  of  1.  5:1  are:  (See  paragraph  5.  3.  ) 


Expected  test  time 


p 

in  multiples  of  MTBF) 

.  10 

17  to  30 

.  20 

8  to  14 

.  30 

3  to  5 

Based  on  this,  it  is  apparent  that  a  reduction  in  the  risk  level  can 
cause  a  significant  increase  in  test  time  which  will,  in  turn,  im¬ 
pact  on  testing  costs  and  schedules. 

The  expected  test  time  can  be  reduced  by  careful  design  of  the 
reliability  test.  In  general,  however,  a  firm  8  risk  is  specified 
at  the  time  0^  is  established,  and  prior  to  the  development  of  the 
other  parameters  of  the  test  plan.  Thus,  P  cannot  be  readily 
"manipulated"  in  designing  the  test  p'?n. 


2 

Handbook  HI 08,  Sampling  Procedures  and  Tables  for  Life  and  Reliability 
Testing  (based  on  Exponential  Distribution),  Office  of  the  Assistant 
Secretary  of  Defense  (Supply  and  Logistics). 
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5.  3  Establishing  the  Specified  MTBF  (0  )  and  Producer's  Decision 
Risk  (a). 

The  minimum  acceptable  MTBF  (6^,  and  consumer's  risk  (8) 
are  normally  established  by  the  Air  Force  as  discussed  above. 
Based  on  these  and  other  considerations  the  Air  Force  usually 
selects  the  Test  Plan  to  be  used.  It  is  then  the  responsibility 
of  the  contractor  to  develop  efficiently  and  economically  con¬ 
ducted  reliability  tests  that  will  assure  veriliration  of  achieved 
reliability  without  degrading  the  risk  level  established  by  the 
Air  Force.  Before  the  test  plan  can  be  developed,  however, 
the  specified  MTBF  (6^)  and  the  producer's  risk  (a)  must  also 
be  established. 

The  specified  MTBF  (B^)  is  the  MTBF  requirement  specified  in 
the  detailed  equipment  specification,  and  must  be  established 
prior  to  beginning  the  detailed  design  engineering  activities. 
Furthermore,  0^  is  directly  related  to  0^  and  8,  as  well  as  to 

the  producer's  risk  (a).  Therefore,  the  reliability  test  plan 
must  be  considered,  at  least  in  a  preliminary  sense,  in  estab¬ 
lishing  Gq, 

The  specified  MTBF  (0^)  is  established  as  a  given  multiple  of  the 
minimum  acceptable  MTBF  ( 9 ^ )  such  that  the  ratio  between  these 
two  values  is  equal  to  a  desired  "discrimination  ratio".  This 
parameter  (  0Q /  e ^ )  is  also  selected  based  on  the  number  of 
equipments,  test  time  (i.  e.  ,  tolerance  time),  and  funds  avail¬ 
able.  In  general,  however,  a  practical  value  c£0q/9j  can  be  estab¬ 
lished  based  on  the  specified  consumer's  risk  (0),  and  by  con¬ 
sidering  the  approximate  test  time  that  will  be  available.  Table 
X- 1  provides  a  rough  estimate  of  the  relationship  between  B^/S^, 

a,  8,  and  test  time  for  test  plans  specified  in  MIL.-STD-781  B. 

The  values  in  this  table  are  not  precise  but  will  provide  a  guide 
for  considering  the  related  parameters  in  establishing  an  appro¬ 
priate  value.  The  final  value  is  established  by  the  Air 

Force  after  making  careful  trade-off  between  G q / 0 ^  and  a,  and 

considering  the  constraints  imposed  by  the  pre-established 
minimum  acceptable  MTBF  (6^)  and  consumer's  risk  (8). 

It  should  be  noted  that,  once  established,  the  0^  / 0 ^  value  cannot 
readily  be  changed.  This  is  because  0q / 0 ^  effectively  fixes  0^ 

with  respect  to  the  specified  0^.  Furthermore,  8^  typically  is 

used  by  the  contractor  as  the  "design  center"  for  the  Detailed 
Specifications  and,  therefore,  must  remain  firm  throughout  the 
detailed  design  program. 
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The  other  parameter,  a,  defined  by  the  Air  Force  is  the  risk  con¬ 
tractors  are  willing  to  accept  with  the  test  plan.  This  value  is 
tentatively  established  when  e  is  determined.  Determination 

of  the  test  plan  should  be  based  on  the  final  trade-offs  involving 
consideration  of  relationships  between  c.,  test  time  (or  sampling 
procedure),  and  accept 'reject  criteria  within  the  c  mstraints  of 
the  previously  fixed  /6  and  r  values. 

5.4  Selecting  the  Reliability  Test  Plan. 

In  general,  a  test  plan  is  defined  by  establishing  the  decision 
risks  (S  and  u.j,  di  rimination  ratio  ( Oq / fj  )  and  the  acce.pt/re- 

jei  t  i  ritcria.  M.IL-STD-781  B  defines  30  test  plans  for  reli¬ 
ability  testing.  Three  of  these,  test  plans  XXVI,  XXVII,  and 
XX  VI 11  are  special  purpose  fixed  length  test  plans  for  reli¬ 
ability  demonstration,  production  verification,  and  longevity 
testing,  respectively,  in  which  the  decision  risks  are  not  de¬ 
fined,  and  which  provide  a  semi- quantitative  or  qualitative 
evaluation  of  equipment  reliability.  The  demonstration  and 
production  verification  tests  (XXVI  and  XXVII)  consider  only 
a  "sped  Id  1"  MTBF  and  assume  the  operating  characteristics 
ol  Test  Plan  l  in  the  stated  accept /reject  criteria.  The  longevity 
test  (XXVU1)  is  entirely  qualitative,  and  is  intended  to  permit  an  eval- 
lation  of  patterns  and  causes  of  failures  occurring  over  an  extended  op¬ 
erating  period.  Test  Plan  XXIX  is  an  equipment  screening  test  not  used 
for  reliability  demonstration. 

The  remaining  26  Test  Plans  (  I  through  XXV  )  provide  a  selection 
of  Probability  Ratio  Sequential  Tests  (PRST),  short  run  high  risk 
PRST,  and  fixed  length  tests  applicable  reliability  tests  such 
as  those  performed  during  Category  1  or  Category  II  test  pro¬ 
grams  . 

5.5  Establishing  the  Accept/Reject.  Criteria. 

Specific  accept /reject  criteria  are  specified  in  MIL-STD- 78 1  B 
for  each  test  plan.  Therefore,  the  accept /reject  criteria  for  a 
reliability  test  is  fixed  with  the  selection  of  the  test  plan. 

In  general,  the  accept/reject  decision  is  based  on  the  number  of 
failures  observed  during  a  test,  and  the  total  operating  time.  For 
fixed  length  tests,  the  decision  is  based  on  the  number  of  failures 
occurring  before  a  pre-established  operating  time.  The  accept/ 
reject  decision  for  a  PRST  test,  however,  is  based  on  experience 
accumulated  during  the  test.  The  test  is  continued  until  a  clear*- 
cut  decision  can  be  made  based  on  the  total  operating  time  before 
a  given  number  of  failures  occur.  As  each  failure  occurs,  the  accum¬ 
ulated  operating  time  is  observed,  and  a  decision  is  made  to  either 
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accept,  reject,  or  continue  testing  until  the  next  failure  occurs. 
This  process  is  repeated  foi  each  failure  in  sequence  until  a 
decision  to  either  a  cept  or  reject  is  made.  If  the  decision 
cannot  be  made  before  some  pre- determined  time,  the  test  is 
terminated  and  a  decision  is  made  as  H  the  test  were  a  fixed 
length  test. 

In  Idition  to  the  quantitati  e  accept /reject  criteria  stated  in 
MI  -STD  /'Ml  B ,  t  list  plan  must  also  state  the  manner  in  which 
the  test  time  is  ccumuiated  during  a  test,  and  should  precisely 
define  '  failures  as  applicable  to  the  test.  In  general,  failures 
are  defined  as  "the  inability  of  a  previously  acceptable  item  to 
perform  its  required  function  within  previously  established 
limits''.  However,  details  involving  failure  criteria  must  be 
tatod  for  tl  <■  specific  equipment  under  test. 

The  test  time  is  the  total  time  the  equipment  is  actually  operated 
during  the  test.  This  time  is  usually  accumulated  by  more  than 
one  equipment,  in  which  case  the  test  time  is  the  sum  of  the  op¬ 
erating  time  accumulated  by  all  equipments  included  in  the  test. 
MIL-STD-781B  specifies  general  procedures  for  determining  the 
quantity  of  equipments  for  a  test.  However,  specific  require¬ 
ments  for  selecting  the  "test  sample  should  be  specified  as  part 
of  the  reliability  lest  plan. 

6.  RELIABILITY  TEST  LEVEL 

The  reliability  tests  of  MIL-STD-781B  are  performed  under  specified 
conditions  of  temperature  and  temperature  cycling,  on-off  cycling, 
input  voltage  cycling,  and  mild  vibration  to  simulate  some  of  the  more 
common  stresses  of  the  actual  operating  environment.  A  particular 
combination  of.  such  stresses  defines  a  "test  level"  to  be  maintained 
during  a  reliability  test. 

MIL-STD-781B  outlines  ten  test  levels  that  are  considered  as  mini¬ 
mum  requirements  for  equipments  intended  for  various  applications. 
These  standardized  test  levels  should  be  modified  as  necessary  to 
correspond  to  any  extreme  or  unusual  environmental  conditions  for 
which  the  equipment  is  being  designed.  In  cases  where  specific  en¬ 
vironmental  requirements  have  not  been  specified,  the  following  test 
levels  are  suggested: 

Test  Level  A-  1 :  Fixed  ground  equipment  intended  for  permanent 
or  semi-permanent  installation  in  air  conditioned  buildings,  and 
which  will  be  operated  continuously  (e.  g.  ,  operations  room  equip¬ 
ments). 
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T est  Level  A:  Mobile  or  semi-portable  ground  or  shipboard 
equipment  to  be  installed  in  an  air  conditioned  shelter,  and 
which  will  be  operated  intermittently. 

T eat  Level  B:  Similar  to  Test  Level  A,  except  applicable  to 
non-air  conditioned  environment  in  which  the  ambient  temper¬ 
ature  can  reach  a  high  of  40°C  (104°F).  When  higher  temper¬ 
atures  are  anticipated,  Test  Level  C,  (50°C)  or  Test  Level  D 
(65°C)  may  be  specified  in  lieu  of  Test  Level  B. 

Test  Level  E:  Airborne  Equipment  or  other  equipment  ex¬ 
pected  to  be  operated  intermittently  under  rapidiy  varying 
ambient  temperatures.  Test  Level  E  includes  temperature 
cycling  within  the  range  of  -  54°C  (-  65°F)  to  +  55°C  (+131°F). 
More  extreme  temperature  ranges  are  simulated  by  Test 
Level  F  (-54°C  to  +  71°C),  Test  Level  G  (-  54°C  to  +  95°C), 

Test  Level  H  (-  65°C  to  +  71°C),  and  Test  Level  J  (-  54°C  to 
+  i  2  5°C). 

7.  PART  RELIABILITY  TESTING 

The  preceding  discussions  have  been  concerned  with  reliability 
testing  of  equipments  to  determine  whether  a  specified  reliability 
requirement  has  been  met.  Another  type  of  reliability  test  is  that 
of  testing  parts  to  determine  the  failure  rate  of  parts  that  are  being 
selected  for  use  in  Air  Force  equipments.  In  general,  part  testing 
differs  from  equipment  testing  in  that  tests  are  performed  on  test 
samples  consisting  of  a  number  of  parts  selected  at  random  from 
large  production  lots.  Sampling  plans  for  such  tests  can  be  de¬ 
signed  using  the  standard  procedures  of  MIL-STD- 1 05D,  "Sam¬ 
pling  Procedures  and  Tables  for  Inspection  by  Attributes.  " 

The  procedure  for  part  reliability  testing  is  essentially  the  same  as 
the  procedure  for  any  attribute  sampling  inspection.  The  only  dif¬ 
ference  is  that  sample  items  are  tested  for  life  or  survival  instead 
of  some  other  property.  Therefore,  the  probable  distribution  of 
part  failures  must  be  considered  in  selecting  sampling  plans  from 
MIL-STD-  105D.  A  procedure  and  related  table  of  factors  for 
adapting  the  MIL-STD-  105D  sampling  plans  to  acceptance  sampling 
inspection  when  the  item  quality  is  reliability  is  presented  in  Quality 
and  Reliability  Assurance  Technical  Report  TR7.  ^  This  document 


3 

Quality  and  Reliability  Assurance  Technical  Report  TR7,  Factors  and 
Procedures  for  Applying  MIL-STD- 103D  Sampling  Plans  to  Life  and 
Reliability  Testing,  Office  of  the  Assistant  Secretary  of  Defense  (In¬ 
stallations  and  Logistics)  Washington,  D.C,  20301. 


considers  the  Weibull  distribution,  together  with  the  exponential 
distribution  as  a  special  case,  as  the  underlying  statistical  model. 

The  procedures  described  in  TR7  for  performing  past  reliability 
tests  involve  the  following  general  steps: 

(a,.  A  suitable  sampling  inspection  plan  is  selected  from  MIL- 
STD-105D,  using  tables  and  factors  provided  in  TR7. 

(b)  A  random  sample  of  items  of  the  size  specified  by  the 
selected  MIL-STD- 1  05.D  plan  is  drawn  from  the  production 

lot. 

(c)  The  sample  of  items  are  tested  (operate  1  fo,  he  specified 
period  of  time  t. 

(d)  The  number  of  items  that  failed  during  the  test  is  compared 
with  the  number  of  failures  allowed  under  the  selected  MIL- 
STD- 105D  plan. 

(e)  If  the  number  of  failures  is  equal  to  or  less  than  the  accept¬ 
able  number,  the  lot  is  accepted  as  meeting  the  reliability 
requirement.  If  the  number  of  failures  exceeds  the  accept¬ 
able  number,  the  lot  is  rejected. 

In  adapting  the  MIL-STD-  105D  sampling  plans  to  reliability  testing, 
the  usual  Acceptable  Quality  Level  (AQ.L)  is  related  to  a  dimensionless 
ratio  lOOt/p.  This  ratio  relates  the  required  mean  Life  o.'  a  lot  (|i)  to 
a  specified  tes?  truncation  time  (t).  With  reliability  test  plans  selected 
in  terms  of  these  ratios,  the  probability  of  aceptance  will  be  high  for 
lots  whose  mean  life  meets  the  specified  requirement. 

Because  of  the  complexity  of  the  procedures  in  Technical  Report  TR7, 
and  because  of  the  necessity  for  frequent  referral  to  specific  data  in 
MIL-STD- 1  05D,  a  detailed  discussion  of  part  reliability  testing  pro¬ 
cedure  will  not.  be  presented  in  this  notebook.  Complete  information 
concerning  application  of  the  procedure  is  presented  in  TR7  together 
with  some  practical  examples  which  demonstrate  methods  for  evalu¬ 
ating  the  efiectivcness  of  a  proposed  test  plan,  as  well  as  procedures 
for  designing  reliability  test  plans. 

In  general,  test  plans  are  designed  using  a  table  that  equates  AQL 
levels  to  the  portion  of  a  lot  that  would  have  a  life  equal  to  or  less 
than  some  time  t,  relative  to  the  mean  life  (p)  of  the  lot,  when  the 
item  lives  follow  Weibull  distributions  having  various  shape  param¬ 
eters.  Once  established,  the  equivalent  AQL  values  permit  reli¬ 
ability  test  plans  to  be  designed  in  the  same  manner  as  any  MJL-STD- 
1 05D  test  plan. 
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Certain  precautions  should  be  observed  in  irte rpreting  data  resulting 
trom  part  reliability  tests.  A  reliability  test  is  conducted  by  oper¬ 
ating  each  of  a  number  of  parts  for  a  time  duration  that  is  short  in 
relation  to  the  specified  acceptable  mean  life.  The  results  of  the 
test  are  evaluated  with  reference  to  the  number  of  failures  occurring 
during  the  test.  Conclusions  concerning  the  mean  life  of  the  lot  are 
based  on  an  assumed  distribution  of  part  lives. 

None  of  tin-  parts  in  the  test  sample  are  actually  operated  as  long  as 
the  acceptable  mean  life.  Therefore,  the  reliability  test  does  not 
provide  a  measure  of  the  true  mean  life  or  failure  rate  of  the  part 
in  question.  It  does,  however,  provide  a  given  degree  of  assurance 
that  the  mean  life  of  a  particular  lot  is  not  less  than  a  specified  value, 
and  that  a  large  number  of  such  lots  will  not  contain  more  than  some 
small  percentage  (given  by  the  AQL)  of  unacceptable  parts. 

RELIABILITY  TEST  PLAN  DATA 

The  following  documents  contain  alternate  sampling  procedures  and 
tables  applicable  to  reliability  test  plan  design, 

a.  Sampling  Procedures  and  Taples  for  Life  and  Reliability 
Testing  B-med  on  the  Weibull  Distribution  (Mear.  T  i/e 
Crite rion).  Quality  Control  and  Reliability  Technical 
Report  TR3,  Office  of  the  Assistant  Secretary  of  Defense 
(Installations  and  Logistics),  Washington,  D.  C. 

b .  Sampling  Proee d ure s_  and  Tables  for  Life,  and  R el ia Siiity 
T esting  Based  on  the  Weibull  Distribution  (Hazard  Rate 
Crite  rion).  Quality  Control  and  Reliability  Technical 
Report  TR4,  Office  of  the  Assistant  Secretary  Defense 
(Installations  and  Logistics),  Washington,  D. 

c.  Sampling  Pr*.  endures  and  Tables  for  Life  and  Reliability 
Testing  Base  a  on  the  Weibull  Distribution  (Reliable  Life 
Crite  ricu).  Quality  Control  and  Reliability  Technical 
Report  IRQ  Otfice  ■.  f  the  Assistant  Secretary  of  Defense 
(Ins  fa  i'i  h  l  ions  and  Logistics),  Washington,  D.  C. 

d.  Factors  and  Procedures  for  Applying  MIL-STD-  105Q 
Sampling  Plans  to  Life  and  Reliability  Testing,  Quality 
and  Reliability  Assurance  Technical  Report  TR7,  Office 
of  the  Assistant  Secretary  of  Defense  (Installations  and 
Logistics),  Washington  D.  C. 


e.  Sampling  Procedures  and  Tables  for  Life  and  Reliability 
Testing  (Based  on  Exponential  Distribution).  Quality 
Control  and  Reliability  Handbook  (Interim)  HI  08.  Office 

of  the  Assistant  Secretary  c.f  Defense  (Supply  and  Logistics), 
Washington,  D.  C. 

f.  Tests  for  the  Validity  of  the  Assumption  that  the  Underlying 
Distribution  of  Life  is  Exponential.  Office  of  the  Assistant 
Secretary  of  Defense  (Supply  and  Logistics),  Washington,  D. 
(This  serves  as  a  companion  document  to  HI  08.  ) 

g .  Sampling  Procedures  and  Tables  for  Inspection  by  Attribute s 

MIL-STD- 105D.  ~ 
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CHAPTER  11 


RELIABILITY  IMPROVEMENT 


INTRODUCTION 

Reliability  improvement  includes  a  variety  of  engineering  and  design 
techniques  specifically  directed  toward  either  achieving  a  higher  level 
of  reliability  than  previously  had  been  achieved,  or  increasing  the 
assurance  that  an  established  level  of  reliability  will  be  achieved.  In 
the  basic  context,  reliability  improvement  activities  are  concerned 
with  developing  a  system  that  is  less  vulnerable  to  unavoidable  re¬ 
liability  problems. 

Reliability  problems  can  be  identified  as  resulting  from  two  fundamental 
sources:  failure  of  the  material  from  which  the  system  hardware  is 
constructed,  and  failure  or  error  on  the  part  of  the  human  element  of 
the  system.  These  can  be  defined  in  terms  of  identifiable  causes,  such 
as  physical  stresses  that  cause  material  failure,  and  operational  de¬ 
mands  that  contribute  to  human  error.  In  some  instances  it  is  possible 
to  eliminate  or  reduce  a  particular  stress  or  level  of  operational  com¬ 
plexity  to  the  point  that  significant  failure  or  errors  no  longer  occur. 

In  general,  however,  it  is  more  practical  to  identify  and  counteract 
rather  than  to  eliminate  or  reduce  a  cause  of  unreliability.  For  ex¬ 
ample,  it  is  more  common  to  assure  that  an  item  is  capable  of  with¬ 
standing  a  given  stress  than  to  reduce  the  stress. 

Five  distinct  but  interrelated  areas  are  considered  in  reliability  im¬ 
provement.  The  most  elementary  of  these  is  that  of  assuring  that 
piece  parts  from  which  the  system  will  be  constructed  are  capable  of 
reliable  operation  under  given  stress  levels  and  operational  environ¬ 
ments.  The  correlary  of  this  identifies  the  second  area:  i.e.,  assuring 
that  the  operational  and  environmental  stresses  to  be  encountered  do 
not  exceed  those  for  which  the  parts  are  designed.  In  the  event  of  dis¬ 
crepancy  between  piece  part  capability  and  use  demands,  a  third  area 
of  reliability  improvement  is  available.  This  makes  use  of  techniques 
such  as  derating  to  reduce  the  effect  of  relative  stress  levels  arid  pro¬ 
vide  a  safety  margin  and  corresponding  improvement  in  system  reliability. 
Additionally,  the  tecliniques  of  redundancy  provide  means  for  achieving 
an  even  greater  improvement  in  system  reliability  by  providing  alternate 
means  for  achieving  success.  Also,  consideration  of  factors  contributing 
to  operational  error  can  aid  in  reducing  the  effect  of  the  human  element 
in  unreliability. 
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INDIVIDUAL  PART  RELIABILITY 


System  reliability  is  a  direct  function  of  the  reliability  of  each  of  the 
individual  parts  making  up  that  system.  Improvement  in  overall  re¬ 
liability,  therefore,  requires  consideration  of  the  reliability  of  the 
individual  parts,  and  recognizes  the  true  cause  of  unreliability,  i.e., 
the  vulnerability  of  equipment  function  to  individual  part  failure.  Thus, 
the  first  consideration  in  reliability  improvement  is  that  of  assuring 
that  the  individual  parts  are  capable  of  performing  at  the  required 
level  of  reliability. 

2.  1  Part  Failures 

A  part  is  considered  to  have  failed  when  the  operational  character¬ 
istics  of  the  part  have  changed  until  the  system  operation  is  no 
longer  satisfactory.  Within  this  definition,  part  failures  can  be 
divided  into  two  general  categories.  The  first  category  involves 
a  relatively  gradual  change  in  functional  characteristics  of  the 
part  until  tolerance  limits  are  exceeded.  The  second  category 
involves  abrupt  and  drastic  changes  in  functional  characteristics. 
These  two  categories  of  failure  are  often  called  "tolerance" 
failures  and  "catastrophic"  failures,  respectively.  These  terms 
refer  to  the  abruptness  of  failure,  and  not  to  the  relative  effect 
on  equipment  operation.  In  fact,  within  the  meaning  of  the  terms 
as  used  here,  catastrophic  part  failures  need  not  have  catastrophic 
effects  on  system  performance,  and,  conversely,  the  fact  that  cer¬ 
tain  failures  are  referred  to  as  tolerance  failures  does  not  preclude 
their  having  catastrophic  effects  on  the  system's  performance. 

Although  tolerance  and  catastrophic  failures  may  be  similar  in  their 
effects  on  system  performance,  there  is  often  considerable  difference 
in  their  underlying  causes,  or  mechanisms  of  failure.  Therefore 
methods  of  reducing  failures  of  the  catastrophic  type  and  thereby 
improving  reliability,  may  be  different  than  methods  of  reducing 
failures  of  the  tolerance  type. 

Tolerance  failures  are  typically  caused  by  "wear  out"  or  "drift" 
phenomena,  and  concern  the  respective  part  itself.  Reduction  of 
such  failures  is  most  often  accomplished  by  improvement  of  the 
failure  rate  or  drift  rate  of  the  part  in  question.  Some  improve¬ 
ment  also  can  be  achieved  by  reducing  the  vulnerability  of  the 
equipment  to  variation  in  part  characteristics.  Catastrophic 
failures,  on  the  other  hand  are  usually  the  resxilt  of  thermal, 
electrical,  or  mechanical  stresses.  In  most  cases  it  is  necessary 
to  reduce  such  failures  by  employing  specific  equipment  design 
features  to  "protect"  the  part  in  question.  A  thorough  understanding 
of  certain  relationships  between  part  reliability  and  equipment  re¬ 
liability  will  aid  in  recognizing  the  degree  of  reliability  improvement 
that  can  be  obtained  by  reducing  part  failure. 


2.2  Part  Reliability  vs.  Equipment  Reliability 

An  electronic  equipment  is  a  collection  of  parts  physically  and 
electrically  joined  together  in  such  a  manner  that,  collectively, 
they  perform  a  desired  function  or  functions.  If  an  equipment 
is  capable  of  satisfactorily  performing  its  functions  at  some 
point  in  time,  it  will  continue  to  have  that  capability  until  a 
significant  change  occurs  in  the  operating  characterits ics  of 
some  part,  or  a  group  of  parts  within  the  equipment.  Con¬ 
versely,  if  the  equipment  fails,  a  part  or  group  of  parts  within 
the  equipment  will  have  failed.  Therefore,  it  is  apparent  that 
equipment  reliability  is  a  function  of  the  number  of  failures  of 
individual  parts  within  the  equipment. 

The  specific  relationship  between  part  reliability  and  equipment 
reliability  is  usually  complex  and  is  dependent  on  many  factors, 
including  the  functional  configuration  of  the  equipment  and  the 
application  of  redundancy  (alternative  functions  in  the  event  of 
failure).  For  example,  an  equipment  having  redundant  parts 
will  not  fail  until  all  redundant  parts  have  failed.  In  a  case  such  as 
this,  the  relationship  between  part  and  equipment  reliability 
cannot  be  defined  until  the  functional  configuration  of  the  equip¬ 
ment  is  defined. 

In  the  most  elementary  case,  however,  an  equipment  would  not 
include  redundant  parts,  and  each  part  failure  would  result  in  an 
equipment  failure.  If  simultaneous  failures  were  impossible,  it 
is  apparent  that  the  total  number  of  equipment  failures  during  a 
given  period  of  time  would  be  equal  to  the  sum  of  all  individual 
part  failures  during  the  same  time  period,  such  that: 

F  =  F  +F  +.  .  .  +F 
t  1  2  n 


where 


F^  =  the  total  number  of  equipment  failures 
during  a  given  interval  of  time 

F,,  F_,  .  .  .,  F  =  the  respective  total  numbers  of 
12  n 

failures  of  each  of  the  n  parts  in  the  equipment 
during  the  same  interval  of  time. 

This  additive  relationship  between  total  part  failures  and  total 
equipment  failures  is  independent  of  the  distribution  of  failures 
with  time  (i.e.,  pattern  of  failure)  and,  therefore  should  provide 
a  simple  method  for  relating  part  reliability  to  equipment  re¬ 
liability.  However,  it  is  usually  more  meaningful  to  evaluate 


part  reliability  in  terms  of  failure  rate  rather  than  total  failures 
and  thereby  provide  a  measure  that  is  independent  of  any  specific 
interval  of  operating  time.  In  the  general  case,  a  simple  relation¬ 
ship  does  not  exist  between  part  failure  rate  and  total  number  of 
failures  due  to  a  general  tendency  for  failure  rate  to  change  with 
time.  In  many  practical  cases,  however,  use  can  be  made  of  the 
fact  that  the  failure  rates  of  individual  parts  remain  essentially 
constant  over  extended  periods  of  time.  Thus,  the  total  number 
of  failures  expected  to  occur  during  any  time  interval  can  be  cal¬ 
culated  as  the  product  of  the  failure  rate  and  the  duration  of  the 
time  interval  of  interest.  In  such  cases,  the  distribution  of  failures 
with  time  can  be  described  by  the  exponential  distribution  function, 
such  that  the  reliability  of  a  part  is  related  to  its  failure  rate  as 
follows: 


R(t)  =  e 


-Xt 


whe  re 


R(t)  =  the  reliability  (probability  of  survival)  of 
the  part  over  time  t. 

X  =  the  failure  rate  of  the  part 

t  =  the  time  period  of  interest 

The  reliability  of  a  series  system  of  several  parts  is  equal  to  the 
product  of  the  reliabilities  of  the  individual  items,  such  that: 

R  (t)  =  R  (t).R  (t)..  .R  (t) 
s  1  <L  n 


whe  re 


R  (t)  =  the  reliability  of  the  series  system  over 
s  , 

time  t. 


R.  (t),  R  (t).  .  .  R  (t)  =  the  respective  reliabilities  of 

each  of  the  n  parts  making  up 
the  system. 


If  all  failures  are  exponentially  distributed,  i.e.,  all  parts  have  con¬ 
stant  failure  rates,  this  last  expression  becomes: 


Rs(t) 


-X  t 
e  1 


-u 

e  Z 


-X  t 
e  n  , 
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where  X  ,  a  .  ....  X  are  the  respective  failure  rates  of  the 
I  2  n 

individual  items.  This  expression  can  be  re-written: 


R  (t)  =  e 
s 


-t(X  +  X  +  .  . .  +  X  ) 

12  n 


The  quantity  (X  +  X  +  .  .  .  +  X  )  is  equivalent  to  the  sum  of  the 
12  n 

failure  rates  of  all  parts  in  the  system.  A  quantity  Xs,  equal  to 
the  sum  of  the  individual  part  failure  rates,  can  be  substituted  in 
this  expression,  such  that 


R  (t)  =  e 
s 

Thus,  the  reliability  of  a  series  system  made  up  of  several  items, 
each  having  a  constant  failure  rate  can  be  evaluated  by  considering 
the  effective  failure  rate  of  the  overall  system  to  be  equivalent  to 
the  sum  of  the  failure  rates  of  the  individual  parts  making  up  the 
system. 

Most  practical  systems  do  not  take  on  a  simple  series  configuration 
such  as  that  discussed  above.  However,  in  virtually  all  cases,  a 
system  can  be  divided  into  fundamental  sub-elements  that  can  be 
considered  individually  as  simple  series  systems  for  evaluation 
purposes . 

It  is  interesting  to  note  that  an  additive  relationship  between  part 
failure  rates  and  equipment  failure  rates  can  often  be  used,  even 
though  some  parts  do  not  exhibit  a  constant  failure  rate.  For  ex¬ 
ample,  suppose  one  part  in  an  equipment  exhibits  a  non-constant 
failure  rate,  but  does  show  a  number  of  distinct  time  intervals, 
throughout  which  the  failure  rate  is  substantially  constant.  If 
the  failure  rate  of  this  part  over  the  time  interval  t^  is  Xj,  over 

the  time  interval  t£  is  X^,  etc.,  then  the  reliability  of  this  part 
over  the  time  interval  t  where  t  =  t  +  t?  +  t  +  .  .  .  .  is: 


R(t). 


-Vi 


-At 
2  2 

•e  -e 


"Vs 


/'h  VhYhV'  •  •  ’ 


-tXeq 

=  e 
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where 


At  +\  t  +A  t  +  ... 
1  1  2  2  3  3 


Thus.,  the  average  failure  rate  of  the  part  over  the  total  time 
interval  can  be  used  as  the  equivalent  failure  rate,  Aeq.  This 
value  can  now  be  combined  with  the  failure  rates  of  other  parts 
as  if  it  were  constant. 

Tile  foregoing  discussion  has  been  presented  to  indicate  a  funda¬ 
mental  relationship  between  equipment  reliability  and  part  re¬ 
liability,  i.  e.,  that  a  basic  additive  relationship  exists  between 
individual  part  failure  rate  and  total  equipment  failure  rate,  while 
a  basic  multiplicative  relationship  exists  between  individual  part 
reliability  and  equipment  reliability.  These  relationships  provide 
the  foundation  for  the  reliability  models  by  which  reliability  im¬ 
provement  requirements  are  defined  and  evaluated.  Furthermore, 
assessment  of  the  relative  influence  of  individual  part  reliability 
on  equipment  reliability  aids  in  identifying  reliability  problem  areas 
and  provides  a  means  for  evaluating  the  relative  worth  of  planned 
reliability  improvement  actions. 

IMPROVING  PART  RELIABILITY 

As  previously  discussed,  system  or  equipment  failures  are  a  direct 
result  of  part  failures.  Therefore,  a  fundamental  aspect  of  reliability 
improvement  should  be  that  of  assuring  that  individual  parts  are  appro¬ 
priately  reliable  before  expending  excessive  time  and  funds  in  develop¬ 
ing  failure  counteracting  schemes.  This  implies  two  different,  but  re¬ 
lated  areas  of  activity:  (1)  selecting  types  of  parts  that  are  capable  of 
meeting  the  reliability  requirements  and  assuring  that  individual  parts 
actually  included  in  the  hardware  are  capable  of  performing  as  specified; 
and  (2)  employing  methods  such  as  derating  to  obtain  a  "safety  margin" 
for  parts  in  use. 

3.  1  Part  Selection  for  Reliability  Improvement 

Part  selection  involves  two  areas  of  activity.  The  first  concerns 
the  initial  selection  of  the  type  of  part  to  be  used  in  the  end  item. 
This  involves  close  cooperation  between  design  engineering  and 
reliability  engineering  activities  to: 

a.  Identify  ail  reliability-critical  electrical  stresses,  temperature 
extremes,  and  mechanical  stresses  associated  with  the  item  in 
which  the  part  under  consideration  is  to  be  used,  and 


b.  Evaluate  the  specifications  for  alternative  types  of  parts  to 
identify  the  type  that  will  best  withstand  these  stresses. 
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In  the  selection  of  parts  it  is  often  advisable  to  consider  alternate 
classes  of  parts  in  order  to  obtain  a  significant  improvement  in 
reliability.  For  example,  dramatic  reliability  improvement 
through  the  application  of  solid  state  devices  has  been  demon¬ 
strated  in  the  recent  application  of  microelectronics.  The  proper 
use  of  microelectronics  not  only  results  in  the  obvious  reduction 
of  the  size  and  weight  of  electronic  equipment,  but  it  also  provides 
considerable  control  over  and  improvement  of  the  reliability  of 
equipment.  In  fact,  microelectronics  have  been  used  extensively 
for  the  primary  purpose  of  achieving  high  reliability  in  cases 
where  the  other  factors  such  as  reduction  of  size  and  weight  were 
not  specified  as  design  objectives. 

The  second  part  selection  activity  involves  preconditioning  pro¬ 
cedures  for  detecting  and  eliminating  defective  or  "weak"  parts 
from  production  lots.  The  selection  of  potentially  reliable  types 
of  parts  does  not  assure  the  reliability  of  parts  actually  used. 

A  recent  survey  of  available  test  and  operational  data  from  typical 
microelectronic  devices,  for  example,  indicates  failure  rates  for 
similar  devices  ranging  from  0.  05  failures  per  million  hours  to 
well  over  1.0  failures  per  million  hours.  This  wide  range  of 
failure  rates  has  not  been  related  to  device  type,  but  is  primarily 
dependent  upon  the  effectiveness  of  the  manufacturer's  processing, 
inspection  and  manufacturing  controls. 

When  reliability  achievement  is  a  critical  factor  in  system  develop¬ 
ment,  parts  cannot  be  used  "as  received"  from  the  manufacturer. 
Regardless  of  the  manufacturer's  quality  control  program,  some 
defective  or  substandard  parts  are  possible  in  any  production  lot. 
Therefore,  a  "pre-conditioning"  process  for  detecting  and  eliminat¬ 
ing  substandard  parts  is  an  essential  reliability  improvement  activity. 
The  basic  pre-conditioning  process  consists  of  screening  and  pre¬ 
aging  to  remove  weak  and  defective  items  from  manufacturer's  lots 
consisting  primarily  of  good  quality  items,  and  burn-in  (also  called 
aging)  either  before  or  after  installation  in  the  equipment  as  a  basis 
for  final  selection.  The  burn-in  can  be  under  simulated  end-use 
conditions  or  accelerated  environments. 

Figure  11-1  depicts  the  pre-conditioning  process  within  the  life 
cycle  of  the  items.  The  elimination  of  "manufacturing  freaks" 
that  would  exhibit  earlier  failures  than  those  expected  from  the 
product  (i.  e.  ,  screening)  requires  testing  under  stresses  that  are 
precisely  controlled  so  that  weak  items  are  discovered  without 
damaging  good  items. 
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Figure  11-1.  Pre-Conditioning  Process 


Screening  in  this  manrer  identifies  many  items  that  would 
otherwise  go  undetected,  and  therefore  effectively  increases 
the  reliability  of  the  end  item.  Further  improvement  can 
be  realized  by  also  including  a  "burn-in"  process  which 
consists  of  preliminary  operation  ol  the  item  unde"  real 
or  simulated  use,  or  accelerated  environmental  conditions, 
in  order  to  encourage  early  failure  of  weak  items. 

3.  2  Selection  of  Non-Standard  Parts 

To  properly  control  the  extensive  and  complex  part  procurement 
activity,  the  Air  Force  maintains  a  standardization  program 
which  requires  that  parts  meet  military  specifications  and  stan¬ 
dards.  Documents  such  as  the  established  reliability  specifi¬ 
cations  not  only  control  functional  and  physical  characteristics 
of  parts,  but  also  establish  appropriate  part  reliability.  Therefore, 
standardized  parts  meeting  military  established  reliability  speci¬ 
fications  will  exhibit  a  stated  level  of  reliability.  Many  parts, 
however,  are  not  covered  by  military  specifications  and  standards. 
Such  "non-standard''  parts,  are  either  too  limited  in  application 
to  warrant  the  preparation  of  military  specifications,  or  are  too 
newly  developed  for  military  specifications  to  have  been  established. 
Reliability  information  on  non-standard  parts  is  often  limited  to 
the  extent  that  supplementary  data  will  be  required  before  the  part 
can  be  accepted  for  use.  Therefore,  the  selection  of  non-standard 
parts  often  requires  reliability  assurance  procedures  that  may  be 
more  involved  than  those  used  in  selecting  standard  parts. 

The  most  dependable,  as  well  as  the  most  usable,  part  reliability 
information  is  obtained  from  well  conceived  and  carefully  monitored 
reliability  tests.  Other  data  sources,  such  as  manufacturer's 
data  and  past  performance  data,  may  also  be  utilized,  but  the 
validity  of  such  data  may  be  questionable,  and  usually  provides 
only  a  qualitative  indication  of  the  reliability  of  the  part  for  the 
application  under  consideration. 

For  non-standard  parts  having  a  history  of  past  applications,  and 
where  information  has  been  logged  carefully  and  is  sufficiently 
extensive  to  yield  failure  data  at  required  levels  of  confidence, 
a  usable  reliability  prediction  can  be  made.  It  is  essential  when 
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using  such  data,  however,  that  the  applications  referred  to  in  the 
data  should  be  as  nearly  similar  to  the  contemplated  applications 
as  possible.  Also  the  stipulated  electrical  characteristics  of  the 
part  for  which  the  performance  data  are  available  should  fairly 
closely  resemble  the  part  under  consideration.  Unfortunately, 
however,  those  application  details  that  have  been  recorded  are 
rarely  available  in  exactly  the  form  required,  and  any  reliability 
prediction  based  on  them  should  be  used  with  the  knowledge  that 
there  may  be  differences  between  the  part  under  consideration  and 
the  part  used  in  generating  the  data. 

The  most  meaningful  reliability  data  for  non-standard  parts  are 
derived  from  well  designed  and  accurately  monitored  reliability 
tests,  the  results  of  which  should  be  carefully  analyzed  to  de¬ 
termine  failure  rate  or  reliability  characteristics.  Reliability 
data,  however,  are  only  as  sound  as  the  tests  from,  which  they 
are  drawn.  Several  reliability  testing  procedures  are  available 
for  application  in  the  selection  and  acceptance  of  non-standard 
parts . 

The  most  common  and  meaningful  reliability  test  is  the  time- 
oriented  (life  test)  type.  However,  strength  oriented  and  de¬ 
gradation  oriented  types  are  also  useful.  The  latter  types  do 
not  provide  a  quantitative  evaluation  of  actual  reliability  of  the 
part,  but  they  do  help  provide  a  safety  margin  for  unexpected 
stress  peaks  and  are  valuable  when  it  is  economically  unfeasible 
to  determine  the  actual  value  with  a  time-oriented  test. 

Time-oriented  reliability  life  tests  involve  operation  of  the  part 
under  stresses  as  close  as  possible  to  conditions  of  actual  operation. 
The  part  is  permitted  to  operate  in  this  fashion  for  a  predetermined 
time  or  until  failure  occurs.  Following  failure,  the  test  is  con¬ 
tinued  using  another  part  of  the  same  type  until  a  pre-determined 
test  time  or  number  of  failures  has  been  experienced.  The  test 
is  then  halted  and  the  failure  rate  is  calculated  by  statistical  analysis 
of  the  test  data. 

Time-oriented  life  tests  can  be  classified  as  follows; 

a.  Nonvariate  or  static  life  tests,  which  involve  operating  the  part 
under  one  set  of  stress  conditions  for  a  predetermined  time 
period.  Most  tests  in  this  category  are  performed  as  rating 
verification  tests  to  provide  assurai.ee  that  the  part  at  least 
meets  minimum  life  requirements.  As  such,  the  tests  seldom 
provide  the  true  failure  rate  of  the  part.  Determining  failure 
rates  in  this  manner  would  require  either  excessively  large 
sample  sizes  or  long  periods  of  test  time. 
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b.  Univariate  life  tests,  which  measure  the  life  of  a  device  under 
varying  levels  cf  stress.  A  critical  stress  such  as  temper¬ 
ature  is  varied  in  a  sfen-stress  technique  to  obtain  time-to- 
failure  vs.  stress  level  patterns.  A  variation  of  this  technique 
involves  application  of  a  controlled,  continuously  increasing 
stress.  These  tests  are  particularly  useful  for  delineating 

the  stress  levels  at  which  different  failure  modes  occur,  and 
in  correlating  failure  rate  with  stress  levels.  Such  tests  are 
somewhat  expensive  and  time  consuming,  and  are  not  nor¬ 
mally  used  as  acceptance  tests. 

c.  Multivariate  life  tests,  which  subject  the  item  to  a  combination 
of  different  stresses  with  varying  levels  of  intensity.  These 
tests  are  usually  applied  as  product  qualification  or  design 
approval  tests,  as  well  as  in  basic  research  and  development 
studies.  Multivariate  tests  utilize  sophisticated  techniques 

of  statistical  experimental  design,  and  require  the  use  of 
somewhat  complex  statistical  techniques,  such  as  multiple 
regression  analysis,  and  analysis  of  variance.  Multivariate 
life  tests  are  time  consuming  and  usua'ly  require  expensive 
test  and  data  processing  facilities. 

Strength  oriented  tests  are  often  used  to  evaluate  part  reli-ib  tity 
when  it  becomes  impractical  to  run  the  time-consuming  life  tests. 
However,  such  tests  provide  qualitative  information  and  do  not 
provide  quantified  part  failure  rate  data.  This  type  of  testing  pro¬ 
vides  an  indication  of  the  strength  or  ability  of  the  part  to  with¬ 
stand  stresses  resulting  from  quantifiable  environmental  or  elec¬ 
trical  influences.  These  tests  are  performed  by  increasing  stress 
levels  until  failure  occurs.  Statistical  analysis  of  the  stress-to- 
failure  data  provides  information  that  can  be  used  to  establish 
"safety  margins"  between  the  stresses  likely  to  be  encountered 
during  operation  and  the  probable  strength  of  the  part. 

Degradation  oriented  testing  provides  a  measure  of  failures  likely 
to  result  from  steady  degradation  of  electrical  or  physical  charac¬ 
teristics  of  the  part.  A  degradation  failure  occurs  when  the 
characteristic  changes  sufficiently  to  cause  circuit  malfunction. 
Consequently,  failure  by  degradation  is  a  function  not  only  01  the 
part  itself  but  of  the  characteristics  of  the  circuit  within  which  it 
is  used.  By  means  of  a  special  series  of  tests,  it  is  possible  to 
determine  the  expected  characteristics  of  a  part,  the  probable 
variation  of  similar  parts  as  they  are  produced,  and  the  manner 
in  which  the  characteristics  can  be  expected  to  change  with  time. 
Thus,  if  the  circuit  in  which  the  part  is  to  be  used  is  analyzed  to 
determine  the  allowable  part  parameter  tolerance,  the  prospective 
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parts  can  be  selected  in  a  manner  that  will  minimize  degradation 
failures.  The  circuit  analysis  required  to  establish  part  toler¬ 
ance  can  be  performed  with  reasonable  economy.  However, 
initial  testing  to  define  the  degradation  or  parameter  drift  rate 
characteristics  of  part  can  be  time  consuming  and  expensive. 

3.  3  Improving  Part  Reliability  by  Derating 

The  discussion  of  the  preceeding  paragraphs  considered  the  part 
itself  in  improving  reliability,  in  that  the  procedures  described 
are  directed  toward  assuring  that  parts  selected  are  inherently 
reliable  and  capable  of  withstanding  the  stresses  to  which  they 
will  be  submitted.  Additional  improvements  in  part  reliability 
can  be  realized,  however,  by  applying  the  techniques  of  derating 
which  consider  the  functional  design  of  the  equipment  in  which  the 
parts  are  to  be  used. 

Derating,  i.  e,  ,  operating  a  part  at  less  severe  stresses  than 
those  for  which  it  is  rated,  is  effective  because  the  life  of  most 
parts  tends  to  increase  as  the  applied  stress  levels  are  decreased 
below  the  rated  value.  In  general,  derating  involves  either  alter¬ 
ing  a  design  to  reduce  the  stresses  applied  to  an  individual  part, 
or  using  a  part  capable  of  withstanding  higher  stresses  than  those 
present. 

Derating  procedures  vary  with  different  types  of  parts  and  their 
application.  Resistors  and  capacitors,  for  example,  are  de¬ 
rated  by  decreasing  the  ratio  of  operating  electrical  stress  to 
rated  electrical  stress.  As  an  example,  a  resistor  rated  at  4 
watts  will  ue  derated  by  a  ratio  of  0.  5  when  used  in  a  Z  watt  appli¬ 
cation.  Electron  tubes  and  semiconductors,  are  also  derated  by 
keeping  the  power  dissipation  below  the  rated  level.  Other  parts, 
notably  capacitors,  are  derated  by  maintaining  the  applied  voltage 
at  a  lower  value  than  the  voltage  for  which  the  part  is  rated. 

One  procedure  for  derating  of  electronic  parts  involves  the  use  of 
derating  curves  which  usually  relate  derating  levels  to  some  critical 
environmental  or  physical  factor.  Such  curves  typically  are  in¬ 
cluded  with  the  specifications  for  the  part  in  question.  A  typical 
derating  curve  is  illustrated  in  Figure  11-2.  This  curve  indicates 
the  relationship  between  operating  power  derating  ratios  and  maxi¬ 
mum  allowable  ambient  temperature  for  carbon  composition  resis¬ 
tors.  Conversely,  the  derating  curve  also  indicates  the  minimum 
amount  of  derating  necessary  before  a  part  can  be  operated  at  a 
given  ambient  temperature.  This  curve  only  indicates  the  amount 
of  derating  necessary  to  preclude  degrading  the  reliability  of  the 
part.  This  does  not  quantify  the  reliability  improvement  that  will 
be  achieved  by  additional  derating. 
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More  precise  evaluation  of  reliability  improvement  resulting 
from  derating  can  be  obtained  from  failure  rate  vs.  applied 
stress  curves  as  used  in  reliability  prediction.  For  example, 
the  failure  rate  vs.  applied  stress  curves  used  in  reliability 
prediction  relates  stress  ratio  and  ambient  temperature  levels 
to  resulting  failure  rate.  Thus  the  results  of  derating  to  any 
given  level  can  be  quantified  in  terms  of  reliability  improve¬ 
ment.  The  failure  rate  curves  in  Volume  II  of  this  notebook 
are  typical  of  such  curves. 

One  disadvantage  of  derating  is  that  any  improvement  in  re¬ 
liability  is  usually  accompanied  by  an  unavoidable  increase  in 
either  the  total  number  or  physical  size  of  parts  used.  Thus,  it 
it  apparent  that  engineering  decisions  concerning  part  derating 
will  involve  certain  trade-off  analyses  to  weigh  the  improvement 
in  reliability  against  the  associated  increases  in  weight,  volume, 
and  possible  cost. 

For  example,  derating  of  composition  resistors  usually  involves 
an  increase  in  the  physical  size  of  the  resistor.  Therefore,  trade¬ 
off  decisions  may  be  required  to  weigh  relative  reliability  improve¬ 
ment  against  increasing  size. 

A  typical  derating  trade-off  curve  representative  of  such  relation¬ 
ships  is  presented  in  Figure  11-3.  This  figure  indicates  that  use 
of  a  resistor  of  greater  than  1/2  watt  rating  in  a  1 /10-watt  appli¬ 
cation  will  probably  not  be  justified,  especially  if  volume  is  a 
critical  factor. 

Trade-off  between  reliability  and  weight,-  volume  or  some  other 
variable  often  requires  quantification  of  the  failure  rate  reduction 
per  unit  change  in  the  end  item  weight  or  volume.  One  such  measure 
is  the  derating  figure  of  merit  (dfm)  which  simultaneously  relates 
changes  in  failure  rate,  with  associated  changes  in  weight,  and  vol¬ 
ume.  Such  a  dfm  is  mathematically  expressed  as: 


WiAV  +  AWV[f  AWAY 


where 


AX  is  the  failure  rate  reduction  due  to  an  incremental 
step  in  derating. 

AW  is  the  change  in  weight  due  to  the  use  of  a  higher 
rated  part. 
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hV  is  the  change  in  volume  due  to  the  use  of  a  higher 
rated  pa rt. 

W.  is  the  initial  total  weight  of  all  parts  that  are  con¬ 
sidered  for  derating. 

V.  is  the  initial  total  volume  of  all  parts  that  are  con¬ 
sidered  for  derating. 

It  may  be  necessary  in  many  designs  to  obtain  maximum  reliability 
within  specified  limits  of  weight  and.  volume.  In  such  instances  all 
parts  to  be  derated  must  be  evaluated  from  an  overall  design  point 
of  view  to  select  the  best  combination  of  parts  to  be  derated  and  the 
level  to  which  each  should  be  derated.  The  dfm  provides  means 
for  making  a  comparison  of  the  changes  in  failure  rate,  weight, 
and  volume  of  many  parts,  and  thus  de  tines  the  combination  of 
parts  to  be  derated  and  the  level  to  which  each  part  should  be  de¬ 
rated  to  provide  the  greatest  improvement  in  reliability  of  an 
equipment  with  a  minimum  increase  in  the  physical  dimensions  of 
the  equipment., 

4.  ENVIRONMENTAL  STRESS  CONSIDERATIONS  IN  RELIABILITY 
IMPROVEMENT 

Up  to  this  point,  this  chapter  has  been  concerned  with  methods  for 
improving  reliability  by  assuring  that  parts  selected  for  use  in  equip¬ 
ments  are  inherently  reliable,  and  by  assuring  that  stresses  created 
by  the  equipment  do  not  seriously  degrade  the  potential  reliability  of 
the  parts.  Further  improvement  in  equipment  reliability  can  be  realized 
by  recognizing  and  counteracting  certain  degrading  stresses  created  by 
the  environment  in  which  the  equipment  will  be  operated.  In  fact,  ignor¬ 
ing  chop 4 actors  in  system  design  can  result  in  a  complete  compromise 
of  t/ie  potential  reliability  level. 

The  importance  of  considering  environmental  factors  in  system  design 
has  been  recognized  foi  many  years,  and  concerted  efforts  to  solve  the 
problems  were  initiated  long  before  ''r el iabiiity  engineering"  was  recog¬ 
nized  as  a  separate  discipline.  This  effort  has  resulted  in  the  develop¬ 
ment  of  standardized  procedures  such  as  those  established  in  MIL-STD- 
8lO(USAF)  for  testing  the  ability  of  an  item  to  withstand  the  deleterious 
effects  of  environments  peculiar  to  military  operations.  Such  testing 
requirements  are  now  imposed  as  part  of  the  quality  assurance  and 
acceptance  testing  provisions  of  virtually  all  system  and  end  item 
spec  ifications . 

An  important  aspect  in  reliability  improvement  is  the  identification  and 
accurate  description  of  the  environments  to  which  the  equipment  will  be 
subjected,  consideration  of  the  manner  in  which  the  various  environmental 
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stresses  effect  equipment  reliability,  and  development  of  appropriate 
objectives  for  the  equipment  engineering  activities.  This  involves 
considering  a  variety  of  atmospheric,  physical  force,  and  radiation 
factors  that  characterize  the  use  environment. 

4.  1  Atmospheric  Stresses  That  Degrade  Reliability 

The  following  discussion  highlights  some  of  the  characteristics  of 
the  environmental  atmosphere,  which  can  seriously  degrade  equip¬ 
ment  reliability. 

a.  Temperature  Extremes.  The  range  of  ambient  temperatures 
expected  in  ground  operations  is  -65°  to  +  1.-5°F.  Even  greater 
extremes  are  encountered  in  world-wide  operation  and  storage. 
For  example,  surface  transportation  and  storage  temperature 

as  low  as  -80°  F  can  be  expected,  while  temperatures  in  densely 
packed  electronic  equipment  may  reach  400°  F  in  the  vicinity  of 
tubes.  These  temperatures  have  relatively  little  effect  on  most 
metals  and  ceramics,  but  can  greatly  effect  the  physical  prop¬ 
erties  of  lubricants,  plastics  and  other  organic  materials. 

In  an  equipment,  high  temperature  conditions  may  cause  the 
permanent  set  of  packings  and  gaskets.  Binding  of  parts  may 
also  result  in  items  of  complex  construction  due  to  differential 
expansion  of  dissimilar  metals.  Rubber,  plastic,  and  plywood 
may  tend  to  discolor,  crack,  bulge,  check  or  craze.  Closure 
and  sealing  strips  may  partially  melt  and  adhere  to  contacting 
parts.  At  the  opposite  extreme,  2.  few  of  the  difficulties  as¬ 
sociated  with  low  temperatures  are  differential  contraction  of 
metal  parts,  loss  of  resiliency  of  packings  and  congealing  of 
lubricants . 

b.  Thermal  Shock.  Additional  damage  can  result  from  a  sudden 
change  in  temperature,  even  though  the  extremes  in  temper¬ 
ature  mentioned  above  may  not  be  reached.  Such  thermal 
shock  can  be  encountered  during  rapid  altitude  changes  dur¬ 
ing  aerospace  service,  or  while  an  equipment  is  being  trans¬ 
ported  from  one  temperature  extreme  to  another  during  ground 
service.  A  typical  temperature  shock  test  would  involve  temper¬ 
ature  changes  from  -40°  F  to  +185°  F  within  a  time  span  of  5 
minutes . 

Effects  of  thermal  shock  include  cracking  and  delamination  of 
finishes,  cracking  and  crazing  of  embedding  and  encapsulating 
compounds,  opening  of  thermal  seals  and  case  seams,  leakage 
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of  filling  materials,  and  changes  in  electrical  cnaracter- 
istics  due  to  mechanical  displacement  or  rupture  of  con¬ 
ductors  or  of  insulating  materials. 

c.  Low  Pressure.  Damaging  effects  of  low  pressure  include 
leakage  of  gases  or  fluids  from  gasket  sealed  enclosures 
and  rupture  of  presurized  containers.  Under  low  pressure 
conditions  low  density  material  tend  to  sublime  and  many 
materials  change  their  physical  and  chemical  properties. 

In  addition,  erratic  opera,  ion  or  malfunction  of  equipment 
may  result  from  arcing  or  corona,  and  greatly  decreased 
efficiency  of  convection  and  conduction  as  heat  transfer 
mechanisms  under  low  pressure  conditions  increase  the 
high  and  low  temperature  problems. 

In  general,  the  low-pressure  extreme  is  in  the  order  of  3.44 
inches  of  mercury  (equivalent  to  50,  000  feet  altitude).  How¬ 
ever,  in  counteracting  the  effects  of  low  pressure,  the  design 
engineer  may  often  be  concerned  with  pressures  such  as  those 
experienced  at  altitudes  as  high  as  100,  000  feet  and  even  higher, 
with  the  limit  being  the  nearly  total  vacuum  of  space. 

d.  Humidity.  High  Humidity,  especially  when  combined  with  high 
temperature  is  another  atmospheric  problem  encountered  by 
che  reliability  engineers.  Corrosion  is  the  most  common  effect 
of  humidity.  However,  hygroscopic  materials  also  are  sen¬ 
sitive  tc  moisture  and  deteriorate  rapidly  under  humid  con¬ 
ditions.  Absorption  of  moisture  by  many  materials  results  in 
swelling,  which  destroys  their  functional  utility  and  causer- 

loss  of  physical  strength  and  changes  in  other  important  mechan¬ 
ical  properties.  Insulating  materials  whim  absorb  moisture 
may  suffer  degradation  of  their  electrical  properties. 

e.  A.tmospiic  :u  -  Contamination.  Salt,  sand  and  dust  contamination 
ir:  the  atmosphere  are  significant  problems  in  land-based  equip¬ 
ment.  Ho r  -example,  no  metal  is  immune  to  the  effects  of  salt 
air.  Salt,  in  the  atmosphere  can  significantly  accelerate  cor¬ 
rosion.  Furthermore,  galvanic  corrosion  can  be  a  serious 
problem  when  any  two  metals  are  in  contact  in  the  presence  of 
salt  and  moisture.  For  this  reason,  protective  devices  such 

as  zinc  ->r  cadmium  coatings  often  fail  in  salt  air  because  they 
form  galvanic  couples  with  the  base  metal. 

On  deserts,  beaches,  volcano  ash  deposits,  or  plowed  fields, 
dust  and  sand  are  a  hazard  to  mechanical  reliability.  Airborne 
dusi  and  fine  sand  can  enter  seemingly  impenetrable  locations, 
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and  accumulate  to  cause  accelerated  wear  in  bearings  of 
shafts  and  motors.  In  addition,  dust  with  a  static  charge 
accumulates  at  points  of  high  potential,  increasing  the  risk 
of  arc-over.  If  the  dust  is  hygroscopic  or  if  the  humidity 
is  high,  the  water-dust  mixture  forms  an  ionized  path  for 
arc-over , 

f.  Breathing.  Alternate  day  heating  and  night  cooling,  or  alter¬ 
nate  increasing  and  decreasing  of  atmospheric  pressure,  can 
cause  an  intake-exhaust  cycle  known  as  breathing.  This  can 
result  in  an  acceleration  of  the  effects  of  humidity  and  atmos¬ 
pheric  contamination  and,  therefore,  is  an  important  con¬ 
sideration  in  reliability  improvement. 

4.  2  Physical  Stresses  That  Degrade  Reliability 

In  addition  to  the  characteristics  of  the  atmosphere,  the  operating 
environment  also  includes  certain  physical  stresses  which  must  be 
considered  in  any  reliability  improvement  program.  The  more 
significant  of  these  are  the  shock  and  vibration  forces  encountered 
during  transportation  and  handling  as  well  as  during  operation. 

The  first  problem  in  reducing  damage  due  to  shock  and  vibration 
is  that  of  determining  the  nature  of  the  forces  that  will  be  encountered. 
This  involves  a  careful  study  of  the  use  environment  and  can  involve 
prediction  of  a  wide  range  of  complex  characteristics  such  as  ampli¬ 
tude  and  duration  of  shock,  or  frequency  and  amplitude  of  vibration. 

In  addition  to  defining  the  physical  forces,  an  even  more  difficult 
problem  is  encountered  in  determining  their  effect  on  the  equip¬ 
ment,  and  in  developing  means  for  counteracting  these  forces. 

This  can  involve  extensive  engineering  analyses  to  assess  moments 
of  inertia,  resonant  frequencies,  and  other  characteristics  of  a 
proposed  design,  predicting  how  the  items  will  react  to  the  external 
forces,  and  devising  methods  for  preventing  damage.  Such  mechanical 
engineering  activities  are  not  defined  as  part  of  reliability  improve¬ 
ment,  but  are  essential  to  the  total  reliability  of  the  system. 

4.  3  Radiation  Phenomena  That  Degrade  Reliability 

Electromagnetic  and  nuclear  radiation  are  some  additional  factors 
that  must  be  considered  in  improving  system  reliability  in  the  use 
environment.  Even  though  the  effect  of  the  two  types  of  radiation 
is  quite  different,  both  can  cause  degradation  in  the  operational 
effectiveness  of  a  system  and,  therefore,  must  be  considered  as 
factors  in  reliability  improvement. 
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a.  Electromagnetic  Radiation.  Electromagnetic  radiation  does 
not  cause  physical  damage  and,  therefore,  is  not  a  factor  in 
the  ''hardware"  reliability.  However,  interference  due  to 
both  natural  and  man-made  radiation  sources  often  degrades 
the  operational  effectiveness  of  electronic  systems  to  the 
extent  that  serious  "operational"  reliability  problems  are 
experienced.  Therefore,  the  identification  of  possible  sources 
and  characteristics  of  unwanted  electromagnetic  radiation 

and  counteracting  the  adverse  effects  of  such  radiation  is  an 
important  part  of  the  overall  reliability  improvement  activity. 

Interference  can  be  caused  by  any  unwanted  signal  or  random 
noise  originating  from  a  variety  of  terrestrial  and  extra¬ 
terrestrial  sources.  For  example,  electromagnetic  noise  is 
generated  by  natural  sources  such  as  lightning  and  aurora 
activity,  and  interference  from  solar  and  celestial  sources  is 
not  uncommon.  In  addition,  automobile  ignition,  electrical 
machinery  and  high  tension  transmission  lines  are  typical 
sources  of  unavoidable  man-made  interference.  Also,  noise 
is  generated  within  the  disturbed  equipment  itself.  This  in¬ 
cludes  random  "shot  effect"  noise  from  electron  tubes,  resistor 
noise,  motor  noise  and  component  microphonics. 

Methods  for  reducing  the  effect  of  electromagnetic  noise  inter¬ 
ference  are  as  varied  as  are  the  sources  and  characteristics  of 
the  noise.  Therefore,  interference  reduction  will  involve  many 
different  engineering  activities,  ranging  from  development  of 
input  filters  and  shielding  devices  to  development  of  superior 

piece  parts  and  basic  material.  (See  Reference  11  at  the  end  of 
this  section.  ) 

b.  Nuclear  Radiation.  Unlike  electromagnetic  radiation,  nuclear 
radiation  can  cause  physical  damage  to  equipment  items  and 
material  and,  therefore,  is  a  direct  consideration  of  the  relia¬ 
bility  improvement  program.  Radiation  can  cause  temporary 
or  permanent  damage  to  many  types  of  electronic  parts,  pri¬ 
marily  by  affecting  organic  materials  used  in  insulation  and 
dielectrics.  Certain  inorganic  compounds  are  also  affected 
where  alteration  of  the  atomic  or  molecular  configuration  will 
degrade  performance.  Some  typical  effects  of  nuclear  radiation, 
on  electronic  devices  are: 

.  Temporary  or  permanent  alteration  of  semiconductor 
device  characteristics  due  to  production  of  extra  car¬ 
riers  by  gamma  radiation,  and  alteration  of  the  atomic 
structure  of  the  semiconductor  material  by  fast  neutron 
bombardment. 
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.  Alteration  of  the  characteristics  of  resistance  elements 
of  variotis  type  of  resistors.  Significant  damage  due  to 
high  energy  radiation  can  occur  in  carbon  composition, 
film  and  wirewound  resistors.  This  damage  is  primarily 
due:  to  deterioration  of  organic  insulating  or  binding 
material. 

.  Temporary  or  permanent  damage  to  transformers  due  to 

damage  to  organic  insulating  material,  ionization  of  gasses 
which  :educe  insulation  resistances  and  increase  arc-over 
damage,  and  changes  in  magnetic  properties  of  core  material. 

4.4  Reliability  Problems  in  the  Space  Environment 

Various  factors  affect  the  operation  and  reliability  of  electronic 
equipment  in  space.  Much  is  still  to  be  learned.  However,  some 
phenomena  such  as  low  pressure,  temperature  extreme’  ,  ionization, 
and  Doppler  shifts,  are  known  to  create  serious  reliability  and  op¬ 
erational  problems. 

a. .  Electric  Breakdown.  A  major  effect  of  the  lack  of  gas  pressure 

in  space  is  the  reduction  in  breakdown  voltage.  Breakdown 
voltage  is  also  reduced  in  an  ionized  atmosphere.  It  is  con¬ 
ceivable  that  the  gaseous  regions  surrounding  the  moon  and 
planets  may  both  be  ionized  and  have  reduced  pressure,  and 
that  electric  breakdown  in  these  regions  ard  in  space  will  be 
a  problem. 

b.  Ionization.  Ionization  can  refract,  reflect,  attenuate,  and 
scatter  radio  waves  and,  therefore,  can  affect  operational 
reliability.  The  effect,  however,  decreases  sharply  with  in¬ 
crease  in  frequency.  The  possibilities  of  ionospheres  existing 
around  the  heavenly  bodies,  and  of  generation  of  ion  clouds  by 
cosmic  explosions  and  meteor  bursts,  would  indicate  problems 
in  the  lower  frequency  range.  Above  1,  900  me,  ionization  effect 
should  be  negligible. 

c.  Doppler  Shifts.  From  a  lunar  orbiting  vehicle  to  earth,  the 
Doppler  shift  at  i,  000  me  can  be  as  high  as  100  kc,  and  greater 
for  higher  frequencies.  Thus  band-pass  problems  are  created 
that  would  not  be  a  consideration  in  ground  or  air-based  cn- 

vi  ronments . 

d.  Temperature.  An  inert  body  in  soace  eventually  reaches  a  state 
of  equilibrium  with  its  environment.  A  body  as  close  to  the  sun 
as  Venus  will  have  a  temperature  of  131°F.  At  a  distance  equal 


to  that  of  the  Earth  from  the  sun,  temperature  will  be  43°  F. 
Temperature  will  continue  to  decrease,  with  increase  in 
distance  from  the  sun,  to  the  low  of  -454°  F  in  intersteLlar 
spae  e . 

Reflecting  bodies,  such  as  the  Earth,  can  contribute  to  the 
energy  absorbed  by  bodies  in  space.  The  reflecting  side  of 
the  Earth  will  produce  15  to  30  percent  of  the  total  energy 
absorbed  by  a  body  at  an  altitude  of  1,  000  miles. 

Mechanical  design  to  dissipate  to  space  (the  ultimate  heat  sink) 
the  absorbed  energy  plus  that  body  hoa!  given  off  by  the  cquipmen 
crew  is  a  continuing  problem,  hi  addition,  higher  tempe  ratures 
contribute  to  the  general  noise  level. 

e.  Other  Factors.  Other  factors  that  may  affect  performance  of 
electronic  equipment  in  space  are  meteors  and  meteorites; 
multipath  effects  in  which  reflections  from  space  dust  clouds 
cause  interference  patterns  in  the  signal;  the  time  required 
for  transmission  over  large  space  distances;  and  error  in 
orbiting  paths  and  computations. 

COUNTERACTING  ENVIRONMENTAL  STRESSES 

In  view  of  the  diversity  of  environmental  stresses  that  can  degrade  re¬ 
liability  in  the  operating  environment,  it  is  apparent  that  many  different 
engineering  disciplines  will  be  required  to  develop  protective  or  counter¬ 
acting  design  techniques  that  will  result  in  improved  reliability.  In  many 
cases,  these  activities  will  not  involve  specific  activities  identifiable  as 
"reliability  engineering.  "  However,  the  total  engineering  effort  in  re¬ 
ducing  the  effect  of  adverse  environmental  stresses  is  directed  toward 
the  single  objective  --  improvement  of  operational  reliability. 

Discussion  of  the  various  techniques  for  counteracting  environmental 
stresses  is  outside  the  scope  of  this  notebook.  However,  Table  XI- 1 
will  indicate  some  typical  techniques  used  in  counteracting  the  effects 
of  various  types  of  environmental  stresses. 

REDUNDANCY 

The  most  effective  means  of  improving  system  reliability,  beyond  the 
level  achievable  through  selection  of  reliable  parts  and  protecting  these 
parts  from  operational  and  environmental  stresses,  is  through  the  appli¬ 
cation  of  redundancy  techniques. 
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Salt  Air  Corrosion,  galvanic  action,  de~  j  non-metal  protective  covers,  re¬ 
creased  insulation  resistance.  duced  use  of  dissimilar  metals  in 

contact,  hermetic  sealing,  de - 
humidifiers. 


In  reliability  engineering,  redundancy  can  be  defined  as  the  existence 
of  ;nore  than  one  means  for  accomplishing  a  given  function  within  an 
equipment  or  system.  Essentially,  a  system  or  equipment  has  re¬ 
dundant  functions  if,  after  the  failure  of  one  or  more  of  its  elements, 
it  continues  to  perform  at  a  satisfactory  level.  This  implies  that  an 
alternate  method  is  available  to  perform  the  function. 

Redundancy  can  permit  an  equipment  to  exhibit  better  reliability  than 
its  elements,  and  the  effective  reliability  of  a  system  to  be  higher 
than  that  of  any  of  its  equipments,  Hov/ever,  the  degree  of  improve¬ 
ment  is  dependent  on  the  type  of  redundancy  employed  and  the  character¬ 
istic  of  the  particular  system  or  equipment.  Some  of  these  consider¬ 
ations  are  discussed  below,  together  with  an  introduction  to  certain 
mathematical  aspects  in  the  evaluation  of  the  reliability  of  redundanl 
configurations.  Also,  certain  trade-off  considerations  concerning  cost, 
weight  and  space  penalty  are  introduced. 

6. 1  Classes  of  Redundancy 

There  are  two  general  classes  of  system  nr  equipment  redundancy: 

(1)  Natural  Redundancy,  where  a  system's  fragmented  structure, 
and  variety  of  operating  modes  permits  some  acceptable  level  of 
system  performance  even  though  some  failure  or  degradation  has 
occurred,  and  (Z)  Design  Redundancy,  where  duplicate  or  alternate 
elements  are  specifically  designed-in  to  perform  duplicate  functions 
in  the  event  of  a  failure,  and  thus  allow  continued  performance  without 
degrading  equipment  performance.  These  tv/o  general  classes  are 
discussed  more  fully  in  the  following  paragraphs. 

a.  Natural  Redundancy.  Operational  characteristics  of  complex 
systems  often  provide  alternate  methods  for  performing  given 
types  of  operational  functions.  Thus,  many  systems  have  a 
large  number  of  useful  states  which  permit  successful,  although 
degraded  performance  even  though  certain  elements  of  the  sys¬ 
tem  fail . 

One  example  of  natural  redundancy  is  a  radar  system  that 
utilizes  a  PPI  and  a  B  scope,  each  having  a  specific  operational 
function.  For  maximum  performance,  both  types  of  scopes 
must  be  in  an  upstate.  However,  the  two  types  of  scopes  have 
certain  features  in  common  such  that  if  one  scope  should  fail 
the  other  could  compensate  for  the  failed  scope  and  the  system 
would  still  be  operating,  but  at  a  degraded  level. 

Many  possible  areas  of  natural  redundancy  exist  in  most  sys¬ 
tems.  Therefore,  it  is  often  possible  to  achieve  a  significant 
improvement  over  the  basic  reliability  of  the  system  by  simply 
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recognizing  and  utilizing  the  alternate  functions  that  exist. 

This  potential  for  reliability  improvement  should  be  fully 
explored  and  exploited  before  resorting  to  the  more  ex¬ 
pensive  application  of  design  redundancy.  The  full  potential 
for  reliability  improvement  through  the  use  of  natural  re¬ 
dundancy,  however,  can  only  be  realized  by  completely  enum¬ 
erating  all  possible  system  operating  states  and  their  associated 
functions.  Therefore,  one  area  of  reliability  improvement  is 
that  of  describing  the  performance  characteristics  of  the  various 
system  states  and  evaluating  the  true  system  reliability  with 
respect  to  all  potential  alternate  functions.  This  effort  re¬ 
quires  analysis  to  determine  the  following: 

.  Definitions  of  all  performance  requirements  of  the  system. 

.  Complete  functional  configuration  of  the  system  consider¬ 
ing  all  modes  of  operation. 

.  Degradation  analysis  of  the  system  to  determine  allowable 
levels  of  degradation,  and  to  define  the  true  conditions  con¬ 
stituting  a  failure. 

.  Designation  of  the  appropriate  reliaK’lity  model  and  mean 
life  formulae  that  consider  the  results  of  the  degradation 
analysis. 

.  Designation  of  the  failure  and  repair  rates  of  all  units  that 
comprise  the  system  and  using  them  as  input  data  to  solve 
the  model  and  mean  life  formulae. 

Design  Redundancy.  Design  redundancy  includes  all  system  or 
equipment  configurations  in  which  redundant  elements  have 
been  included  specifically  to  improve  reliability.  There  are 
several  types  of  design  redundancy  which  can  be  described 
according  to  (a)  the  operational  state  of  the  redundant  elements 
while  the  system  is  in  operation,  (b)  circuit  configuration,  and 
(c)  the  existence  or  nonexistence  of  decision  and  switching  (DS) 
devices.  Several  basic  redundant  configurations  are  illustrated 
in  Figure  1  1-4.  The  effectiveness  of  each  of  these  configuration 
in  improving  reliability  is  dependent  on  the  manner  in  which 
the  circuit  fails,  as  well  as  the  circuit  configuration.  Some 
general  considerations  in  the  application  of  the  various  re¬ 
dundant  element  configurations  are: 

Parallel  Redundancy.  Parallel  redundancy  (Diagram  A  of 
Figure  1  1-4)  is  used  when  the  dominant  mode  of  element 
failure  is  open  circuits.  If  one  of  the  two  elements  fail  in 
an  open  condition,  (was  removed  from  the  configuration)  the 


A.  Parallel  Redundancy 
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other  could  still  perform  the  required  function.  This 
arrangement  does  not  provide  protection  against,  short 
circuits  and,  actually,  increases  the  failure  probability 
due  tc  short  circuits. 

Series  Redundancy.  Series  redundancy  (Diagram  B  of 
Figure  11-4)  is  an  arrangement  which  provides  protection 
in  the  event  of  "short  circuit"  failure  because  one  element 
could  perform  the  required  function  even  though  the  other 
element  was  short-circuited.  This  arrangement,  however, 
is  vulnerable  to  open  circuit  failures. 

Series -Parallel  Redundancy.  A  s  eries -parallel  redundant 
configuration  (Diagram  C  of  Figure  11-4),  and  variations 
of  this  basic  configuration,  provide  increased  reliability 
under  conditions  in  which  the  basic  parallel  or  series  con¬ 
figurations  are  not  effective.  For  example,  this  con¬ 
figuration  would  be  equally  effective  in  the  event  of  either 
open  -  or  short-circuit  failures. 

Standby  Redundancy.  In  many  instances  it  is  impractical, 
from  an  engineering  viewpoint,  to  use  redundant  config¬ 
urations  such  as  those  discussed  above  in  which  all  re¬ 
dundant  elements  are  permanently  connected  and  oper¬ 
ational  at  all  times.  In  such  cases,  standby  redundancy 
(Diagram  D  of  Figure  11-4)  can  often  provide  a  solution 
to  the  engineering  problem.  In  these  configurations,  an 
element  is  held  in  "standby"  and  switched  in  to  assume 
the  circuit  function  in  the  event  of  failure  of  the  primary 
element. 

The  spare  element  can  be  either  on  (active)  all  the  time  or 
off  (inactive)  when  the  primary  element  is  performing  its 
function.  These  two  types  of  redundancy  include  switching 
elements  that  must  be  considered  in  evaluating  the  effective 
reliability  of  the  overall  configuration. 

Partial  Redundancy.  Partial  redundancy  is  a  special  case 
of  redundancy  wherein  several  parallel  outputs  are  channeled 
through  a  single  series  device  such  as  a  decision-making 
circuit  which  provides  the  required  function  as  long  as  a 
predetermined  number  of  the  parallel  outputs  are  in  agree¬ 
ment.  The  reliability  of  the  series  element  is  a  critical 
factor  in  the  effectiveness  of  this  configuration.  An  ex¬ 
ample  of  partial  redundancy  is  shown  in  Diagram  E  of 
Figure  1  1-4. 


.  Voting  Redundancy.  Another  SDecial  type  of  redundancy 
includes  means  whereby  the  outputs  of  three  or  more  op¬ 
erating  redundant  units  are  compared,  and  any  one  output 
that  agrees  with  the  majority  of  outputs  is  selected.  This 
configuration,  referred  to  as  voting  redundancy,  is  illus¬ 
trated  in  Diagram  F  of  Figure  11-4. 

6.  2  Evaluation  of  Redundant  Configurations 

One  of  the  major  reliability  engineering  activities  during  system 
design  is  th  evaluation  of  redundant  configurations  to  assess  the 
achieved  level  of  reliability  improvement.  Such  evaluations  not 
only  provide  a  quantitative  measure  of  reliability  improvement, 
but  also  provide  comparative  data  essential  to  many  design  trade¬ 
off  studies.  In  general,  evaluation  of  a  redundant  system  con¬ 
figuration  involves  complex  mathematical  models  which  are  unique 
to  the  specific  system  under  consideration.  However,  any  such 
model  is  composed  of  elements  that  can  be  defined  in  terms  of 
elementary  redundancy  configurations,  which  are  used  ?.s  "building 
blocks"  in  developing  the  overall  system  model. 

Redundant  configurations  are  usually  evaluated  in  terms  of  the 
probability  of  survival  (or  probability  of  failure)  of  individual 
elements  of  the  system.  This  approach  permits  the  application  of 
basic  rules  of  probability  in  developing  a  mathematical  model  of 
system  reliability.  In  certain  cases  system  reliability  is  evaluated 
in  terms  of  MTBF  or  similar  parameters.  In  general,  however, 
techniques  for  performing  evaluations  such  as  this  involve  the 
combination  of  failure  distribution  functions  and  are  beyond  the 
scope  of  this  notebook.  Some  of  the  more  elementary  redundant 
configurations,  which  can  be  evaluated  in  terms  of  MTBF  without 
regard  for  failure  distribution  are  discussed  here  for  illustration 
purposes.  Most  of  the  discussion,  however,  is  concerned  with  the 
more  conversational  probabilistic  approach. 

Procedures  for  developing  models  suitable  for  evaluating  some  of 
the  more  common  elementary  configurations  are  discussed  below. 
These  discussions  are  based  on  the  following  assumptions: 

.  The  units  under  consideration  are  composed  of  independent 
elements  whose  operation  can  be  described  in  discrete  terms 
of  "success"  or  "failure". 

.  All  elements  are  continuously  energized  and  switching  devices 
are  either  unnecessary  or  are  effectively  failure-free. 
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Failure  of  any  type  has  no  adverse  effects  on  the  operation 
of  the  surviving  paths  -  e.g.,  precautions  such  as  use  of 
a  fuse  has  been  taken  to  avoid  unit  failure  through  shorting 
if  a  parallel  path  shorts. 

a.  Basic  Series  System  Reliability.  The  method  for  evalu¬ 
ation  of  a  non-redundant  series  system  is  discussed  first 
to  establish  the  basis  for  the  evaluation  of  redundant  con¬ 
figurations.  A  non-redundant  series  system  of  n  elements 
can  be  represented  by  the  follov/ing  diagram: 


The  reliability  of  this  "n-eiement  series  system,  "  under 
the  basic  assumptions  of  independent  element  failures  and 
the  necessity  of  successful  operation  of  all  elements  for 
system  success,  is 

R’Pl-P2 . Pn  (1) 

where  p^,  p^»  - .  • ,  Pn  are  the  reliabilities  (probability  of 

survival)  of  elements  A,,  A  ,  .  .  . ,  A  ,  respectively, 

1  c  n 

If  all  elements  are  identical  with  reliability  p, 

R  =  Pn  (2) 

The  MTBF  of  the  basic  series  unit  can  be  determined  by 
considering  the  system  failure  rate  to  be  the  sum  of  the 
failure  rates  of  the  individual  elements.  The  unit  MTBF 
is  the  recipx’ocal  of  the  system  failure  rate.  Thus: 

n 

x  =y  x.  (3) 

system  L,  i 

i=l 
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where : 


X  -  the  system  failure  rate  . 

system 


X. 

1 


the  failure  rate  of  one  element  of  the 
series  unit. 


n 


the  total  number  of  series  elements  in 
the  unit. 


Taking  the  reciprocal: 


1 

MTBF  =  — -  (4) 


i=l 


If  all  elements  are  equal,  with  failure  rate  X,  then: 


MTBF 


J _ 0 

n  X  n 


(5) 


where : 


8  =  — ■  =  the  MTBF  of  one  element  of  the  unit. 

A. 


b.  Basic  Parallel  Redundant  Configuration.  Two  elements  in 
a  parallel  redundant  configuration  can  be  represented  by 
a  reliability  block  diagram  as  fellows: 


Input 

I 


Output 
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Blocks  A^  and  represent  independent  elements,  either 

of  which  can  perform  the  required  function.  If  the  probability 
of  element  A^  operating  successfully  over  the  specified 

time  interval  is  p^,  and  if  p^  is  the  corresponding  probability 
for  element  A2>  the  probability  of  success  for  the  unit  can 

be  found  by  considering  that  the  unit  is  successful  if  at  least 
A^  or  A;?  is  operable.  Since  both  elements  are  energized, 

the  events  the  operation  of  A|  and  A2  are  not  mutually  ex¬ 
clusive  events  -  i.e.,  both  A^  and  A^  can  occur.  Therefore, 
the  probability  of  success  using  the  additive  rule  for  non- 
mutually  exclusive  events  is: 

R  =  Pl  +P2-Plp2.  (6) 

An  alternate  derivation  can  be  obtained  by  defining  con¬ 
ditions  of  failure.  The  only  way  the  unit  can  fail  is  through 
failure  of  both  elements.  Since  the  operation  of  A^  and  A 2 

are  assumed  to  be  independent  events,  the  probability  that 
both  elements  fail  is  the  product  of  their  unreliabilities.  If 

R  is  the  "unreliability"  or  probability  of  failure  of  both  ele¬ 
ments,  and  (1-pj)  and  ( 1  -p^ )  are  the  respective  probabilities 

of  independent  failure  of  elements  A^  and  A then: 

R=  (i-P]Hi-P2) 

But  since  the  probability  of  either  a  failure  or  a.  success  is 
unity,  R  +  R  =  1,  and 

R  =  l-(l-Pi)(]-p2)  (7) 

Expressions  (6)  and  (7)  are  equivalent,  and  either  can  be 
used  fur  evaluating  the  reliability  of  two  parallel  rodundanl 
e  lenient  s . 

If  the  1  v/o  elements  are  identical  wlih  reliability  p,  ex¬ 
pression  (6),  and  (7)  become: 

2 

R  =  2p-p  ,  and 
R  -  i-(l-pf 

Example:  If  p.  =  p  =  0.  90,  by  the  additive  rule  oJ"  ex- 

X  Ct 

pression  (b):  R  =  0.  90  +  0.  90  -  0.  81  -  0.  99. 
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c. 


Or  by  the  multiplicative  rule  of  expression  (1): 

R  =  1-(1-.  90)(  1  - .  90)  =  1-.01  =  .99 

Multiple  Parallel  Redundant  Configuration.  By  extensioi 
of  expression  (7),  the  general  expression  for  the  reliability 
of  a  unit  with  m  parallel  elements  is: 


R  =  i  -  (i-p,)(i-pj...(i-p  ) 

1  l.  m 

If  all  elements  are  identical  with  reliability  p,  then: 


(8) 


m 


R  =  l  -  (1-p)  • 

The  general  expression  for  the  MTBF  of  a  unit  with  rn 
identical  parallel  elements  is: 


(9) 


or: 


where: 


m 

MTBF  =  -J- 

X  U 


(10) 


i  =  m-k 


m 

MTBF  =  67  r 

L  i 

i=m-k 


(II) 


MTBF  =  the  mean  time  between  failure  of  the  unit. 

X  =  the  failure  rate  of  any  one  of  the  elements 

0  =  the  mean  life  or  MTBF  of  any  one  of  the 

elements . 

k  =  the  maximum  number  of  elements  that 
can  fail  without  failure  of  the  unit. 


In  the  event  that  any  one  element  can  perform  the  total  unit 
function,  the  unit  will  not  fail  until  all  m  elements  fail.  In 
this  case  k  =m-l,  and  expression  (11)  becomes: 


M.T 


m 

[tbf  =  et  f  =  0(7  +  7+  ...  +  — 

U  1  \  1  2  m 


i=l 


(12) 
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Applying  expression  (12)  in  the  special  case  of  two 
identical  parallel  elements,  as  discussed  in  para¬ 
graph  b  above,  would  yield: 


MTBF  =  6(^-  1 


1 


2  1 


3_d 

> 


Sg_r_ies -Parallel  Configurations.  A  s eries -parallel  con¬ 
figuration,  is  a  series  of  n  basic  parallel  units.  The 
reliability  block  diagram  of  such  a  configuration  will  have 
the  form  shown  in  the  figure  below. 


Unil  I 


Unit  2 


Unit  n 


This  is  the  series -parallel  redundant  counterpart  of  an 
n-element  non- redundant  system.  For  each  unit  there 
are  two  possible  paths  for  unit  success.  The  reliability 
of  one  of  the  units  therefore  can  be  determined  using  ex¬ 
pression  (7)  giving: 


r .  =  1  -  q  q 

J  4ljq2j 

where  q  =  1  -p. 


Assuming  that  unit  failure  probabilities  are  independent, 
the  system  reliability  is  the  product  of  unit  reliabiliti 
Applying  expression  (1): 


es. 


R  =  (1-quq21)<l-qi2q22)...U-q,nq2„> 


(13) 


if  all  elements  are  identical,  with  a  reliability  of  p  and 
unreliability  of  q  then: 


an 


R  =  d-q2)n. 


(14) 
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To  illustrate  the  possible  reliability  advantages  of 
redundancy  for  this  simple  model,  assume  a  basic 
circuit  of  three  identica1  elements,  each  with  a  re¬ 
liability  of  0.  8  over  the  time  period  of  interest. 
Without  redundancy, 

R  -  (0.  8) 3  -  0.  512. 

When  each  element  is  duplicated  by  a  series -parallel 
arrangement, 

R  =  (1  -  q")"’  -  (1  -  0. 20)J  =  0.  885. 

General  Series -Par al lei  Configurations.  In  general,  a 
series  of  multiple  parallel  elements  rather  than  two 
parallel  elements  would  have  a  configuration  as  shown 
in  the  figure  below. 


Ur.il  I 


Unit  2 


Unit  n 


By  extending  the  results  given  in  expression  (7), 


n 
—  7  f 


R  ‘  Ti  (1"qii  q2i - q  •)’ 

1  lj  2j  rnj 

j  =  l 


(15) 


where  q. .  is  the  failure  probability  of  the  ith  element  in  the 
jth  unit. 


If  all  elements  in  a  unit  are  identical. 


n 

R  =  j'I(  1-q™)  (16) 

j  =  l 

Further,  if  all  n  units  are  identical,  and  made  up  of  m 
parallel  elements,  each  having  probability  of  failure  q, 
or  reliability  p. 


R  =  ( 1  -  q*~)n  =  ri-(l-p)m]n 


An  expression  for  the  MTBF  of  a  series-parallel  con¬ 
figuration  can  be  derived  by  combining  expressions  (5) 
and  (11),  such  that: 


where: 

6  =  the  MTBF  of  any  one  element  of  the  system 

(all  elements  are  assumed  to  be  identical). 

m  ~  the  number  of  elements  in  any  one  parallel 
unit,  (all  parallel  units  are  assumed  to 
consist  of  an  equal  number  of  elements). 

k  =  the  maximum  number  of  elements  that  can 
fail  in  any  one  parallel  unit  without  causing 
system  failure. 

n  =  the  number  of  units  connected  in  series  to 

make  up  the  system.  (All  units  are  assumed 
to  be  identical). 

6.  3  General  Expressions  for  Evaluation  of  Basic  Redundant  Configurations 

Any  configuration  of  redundant  elements  can  be  evaluated  by  develop¬ 
ing  models  in  the  manner  illustrated  in  paragraph  6.  2.  In  general, 
the  models  for  evaluating  redundant  configurations  can  become  more 
complex  than  the  simplified  examples  of  paragraph  6.  2,  and  will  not 
be  derived  here.  However,  the  reliability  evaluation  expressions  for 
the  general  case  of  each  of  several  basic  configurations  are  presented 
in  Figure  1.1-5.  These  expressions  can  be  tailored  to  particular 
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situations,  and  can  be  used  in  combination  to  define  practical 
active  rediindancy  configurations  providing  failure  mode  (open 
or  short-circuit  failures)  is  not  a  factor. 

Symbols  used  in  Figure  11-5  an  defined  as  follows: 

Symbol  Definition 

R  Effective  reliability  (probability  of  survival) 

of  the  overall  configuration. 

r  Effective  reliability  of  a  defined  portion  of 

the  overall  configuration. 

p  Reliability  of  a  basic  element. 

q  Probability  of  element  failure  (q  =  1-p). 

MTBF  Mean  Time  Between  Failures  of  the  overall 

configuration. 

W  Mean  Time  Between  Failures  of  any  one  of  a 

number  ol  identical  elements  in  the  configuration. 

.  4  Consideration  of  Open-and  Short-Circuit  Failures 

The  previous  redundant  models  were  based  on  the  assumption  that 
individual  element  or  path  failure  has  no  effect  on  the  operation  of 
surviving  paths.  However,  this  assumption  is  not  always  valid. 
Consider,  for  example,  a  simple  parallel  unit  composed  of  two  ele¬ 
ments,  A  and  B,  each  of  which  can  fail  by  either  open  circuit  failure 
or  short-circuit  failure.  Short  circuit  failure  of  either  of  the  two 
elements  will  "short  out"  the  other  element  and,  therefore,  result 
in  unit  failure.  Therefore,  in  many  practical  cases,  evaluation  of 
redundant  configuration  is  complicated  by  the  necessity  of  consider¬ 
ing  the  type  of  failure.  Methods  for  considering  open  and  short-cir¬ 
cuit  failures  in  evaluating  redundant  configurations  are  reviewed 
below. 

a.  Reliability  of  Basic  Parallel  Configurations.  I'or  two  elements 
in  the  active-parallel  redundant  configuration  below. 


the  unit  will  fail  if  either  of  the  following  events  occur: 


(1)  either  A  o_r  B  shorts,  or 

(2)  both  A  and  B  open. 

The  respective  probabilities  of  these  two  events  are: 


(1)  P(A  or_  B  shorts  )  -  q 


sa 


'sb  ^sa^sb 


=  1  -  (l-q  )(l-q  i  ) 

sa  sb 


(2)  P(A  and  B  open)  ~  q  q  , 
-  oa  ob 


where  q  is  the  probability  that  element  i  opens  and  q  .  is  the 


s  1 


probability  that  element  i  shorts.  Since  events  (1)  and  (2)  are 
mutually  exclusive,  the  probability  of  unit  failure  (P(F))  is  the 
sum  of  the  two  event  probabilities,  or, 

P(F)  =  P(A  o_r  B  Short)  +  P(A  and  B  Open) 

=  1  -  (l-q  )(l-q  ,  )  +  q  Cj  . 

sa  sb  oa  Job 

R  =  l-P(F),  therefore, 

R  =  (1  -q  )( 1  -q  )  -  q  q 

sa  sb  oa  ob 

In  general,  if  there  are  in  parallel  elements, 


m 


m 


R  -  I  i  (i-q  :  q  .  ; 

II  SI  I  I  Ol 

i= 1  i=l 


Further,  if  all  m  elements  are  identical: 


m  m 

R-'1-qs»  -qc 


b.  Reliability  of  Basic  Series  Redundant  Configuration.  In  general, 
a  series  redundant  system  will  fail  if  one  or  more  elements 
are  open-circuited,  or  if  all  elements  are  short-circuited.  For 
example,  in  the  two-element  series  unit  below: 


a 


B 


-o 
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the  unit  will  fail  if  either  of  the  following  events  occur: 

(1)  Both  A  and  B  short,  or 

(2)  Either  A  or  B  oper.j. 

(Note  that  this  is  opposite  from  the  definition  of  failure  for 
a  two-element  parallel  unit.  ) 

A  derivation  similar  to  that  for  the  parallel  case,  but  con¬ 
sidering  conditions  of  failure  for  the  series  redundant  con¬ 
figuration  gives: 


R  =  (1-q  )( 1  -q  )  -  q  q 

oa  ob  sa  sb 

In  general,  if  there  are  n  series  elements: 


n 


n 


R  =/rf  (1-q  .)  -  if  q  . 

I  I  oi  I  l  ^si 

i=l  i=l 


If  all  n  elements  are  identical: 


„  ,n  n 

R  =  (l-qQ)  -  qs 

c.  Optimum  Number  of  Parallel  or  Series  Elements.  The  ex¬ 
pressions  derived  above  for  considering  the  effect  of  short- 
and  open-circuit  failure  on  the  reliability  of  redundant  ele¬ 
ments  indicate  that  an  optimum  number  of  parallel  or  series 

elements  exists  for  various  values  of  q  and  q  .  This  is  illus- 

s  o 

trated  in  Table  XI-2,  where  the  parallel  expression,  R  =  (l-q 

m  ® 

q  ,  is  solved  for  various  values  of  q  and  m  when  q  =  0,  10. 
m  s  o 

Ta.ble  XI-2.  Values  of  R  for  q  =  0.  1 

o 


m 


m 

qs  =  ° 

q  =  0,  05 
s 

q  =  0.  10 
s 

q  =  0.  20 
s 

1 

0.  900 

0.  85 

0.  80 

0.  70 

2 

0.  990 

0.  89 

0.  80 

0.  60 

3 

0.  999 

0.  86 

0.  73 

0.  51 
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PARAllxl  UNIT 


qs  =  probability  of  short-circuit  failure  of  one  element, 
qo  -  probability  of  open- circuit  failure  of  one  element. 


Figure  11-6.  Optimum  Number  of  Redundant  Elements  as  a 

Function  of  Failure  Mode  Probabilities  (Reproduced 
from  NAVWEPS  00-65-502) 
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In  the  case  where  q  =0,  unit  reliability  increases  as  the 

number  of  parallel  elements  is  increased.  In  this  case  R 

would  continue  to  increase  as  m  is  increased.  In  the  case 

where  q  -  0.  05,  however,  R  increases  as  m  is  increased 
s 

from  1  to  2,  but  decreases  as  m  is  further  increased  to  3. 
Therefore,  m  =  2  (i.  e. ,  two  elements  in  parallel)  will  pro¬ 
vide  maximum  reliability  when  q  =0.1  and  q  =0.  05,  but 
3  or  more  units  in  parallel  would  decrease  the  unit  reliability. 

In  the  case  where  a  =  0.  10  parallel  redundancy  will  not  in¬ 
crease  reliability,  and  when  q^  =  0.20  (i.e.,  q  >q  ),  any 

parallel  redundancy  will  actually  decrease  reliability.  In 
the  latter  case,  however,  reliability  could  be  increased  by 
using  series  redundancy. 

Figure  11-6  presents  the  general  solution  of  the  parallel  and 
series  reliability  expressions  for  optimum  numbers  of  re¬ 
dundant  elements,  while  the  relationships  between  q  and  q 

are  varied.  When  q  is  greater  than  q  ,  the  curves  are  entered 

c  s 

via  the  left-hand  and  bottom  scales  to  determine  the  optimum 

number  of  parallel  elements.  When  q  is  greater  than  q  ,  the 

s  °  UD 

curves  are  entered  via  the  top  and  right-hand  scales  to  determine 

the  optimum  number  of  series  elements.  If  q  -  q  ,  then  no  in- 

s  o 

crease  in  reliability  is  possible  without  resorting  to  more  com¬ 
plex  schemes  such  as  series-parallel  redundancy. 

d.  Series-Parallel  and  Parallel-Series  Configurations.  The  re¬ 
liability  of  serie s -parallel,  and  parallel- series  configurations, 
when  short-and  open-circuit  failure  modes  are  considered  and 
where  all  elements  are  equal,  are  determined  by  the  following 
general  expressions. 


Series-Par-  llel 


Unit  4  Unit  9 


R.p*  -  n-U-qs)m]n 


m  =  number  of  parallel  elements 
in  each  unit 

n  =  number  of  units  in  series 


Parallel-Series 


R 

ps 


n 


-  n  nim  n  m  \nim 

-  [i-qs]  -  L i -{ i -qQ)  ] 

=  number  series  elements  in 
each  path 


m 
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=  number  of  parallel  paths 


6.  5  Redundancy  Involving  Switching 

To  this  point,  it  has  been  assumed  that  devices  for  detecting 
element  failure  and  switching- in  redundant  elements  are  either 
unnecessary  or  failure-free.  However,  many  practical  con¬ 
figurations  involve  switching  elements  which  have  some  prob¬ 
ability  of  failure. 

In  general,  a  switching  device  is  effectively  an  element  in  series 
with  a  redundant  unit.  However,  the  expressions  for  evaluating 
redundancies  involving  switching  are  complicated  by  the  necessity 
for  considering  three  general  types  of  switching  failures.  These 
are: 

Dynamic  failure  -  failure  to  switch  when  required 

Static  failure  -  inadvertent  or  premature  switching 

Contact  failure  -  inability  of  the  switch  to  maintain  a  good 
connection. 

Dynamic  switching  failure  always  causes  system  failure;  static 
failure  causes  system  failure  if  the  duplicate  element  has  failed 
(assuming  switching  is  only  in  one  direction);  and  contact  failure 
always  causes  system  failure. 

Time- dependent  situations  must  be  discussed  before  switching 
reliability  can  be  treated  fully;  however,  a  simple  example  will 
prove  fruitful  in  illustrating  the  effects  of  switching  failure  on  re¬ 
dundancy  applications. 

Consider  the  following  two-path  parallel  system  which  requires  a 
decision  and  switching  device  and  which  remains  latched  to  B  once 
B  is  energized  by  the  switch. 
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Three  possible  states  that  may  lead  to  system  success  are: 

State  1:  A  and  B  are  successful  (AB). 

State  2:  A  succeeds,  B  fails  (AB). 

State  3:  A  fails,  B  succeeds  (AB). 

State  1  requires  no  contact  failure  (dynamic  failure  can  occur  only 
if  A  fails;  and  a  static  failure  in  this  case  does  not  result  in  system 
failure). 

State  2  requires  no  contact  failure  and  no  static  failure. 

State  3  requires  no  contact  failure  and  no  dynamic  failure  (static 
failure  cannot  occur  if  A  fails). 

Let: 

p.  =  element  reliability  (i=a,  b;  q.  =  1  -  p . ) ; 

p^  =  conditional  dynamic  reliability  (switching  when  required); 

p  =  conditional  static  reliability  (no  switching  when  not 
required); 

p  =  contact  reliability. 

Consideration  of  each  state  will  provide  an  expression  for  probability 
of  success  as  follows: 


R  =  p  p  p  +  p  q  p  p  +  q  p  p  p  , 
3.  b  c  abet  abcc 


State  1 


State  2 


State  3 
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For  failure-free  switching  (p  =  p'  =  1.  0),  R  =  p  +  2pq.  This  is 

greater  than  nonredundant  reliability,  p,  and  the  redundancy  appli¬ 
cation  therefore  increases  reliability.  However,  if  p  and  p'  are 
less  than  1.0,  the  relationships  between  p,  p  ,  and  p1  become  im¬ 
portant  for  determining  if  redundancy  is  beneficial  and  at  what  level 
it  should  be  introduced. 

6.  6  Voting  Redundancy 

The  following  figure  shows  three  elements.  A,  B,  and  C,  and  the 
associated  switching  and  comparator  circuit  which  make  up  a  typical 
voting  redundant  system. 


(From  NAVWEPS  00-65-502) 


|Thrce-Elemenf  Voting  Redundancy 

The  circuit  function  will  always  be  performed  by  an  element 
whose  output  agrees  with  the  output  of  at  leastone  of  the  other 
elements.  At  least  two  good  elements  are  required  for  successful 
operation  of  the  circuit.  Two  switches  are  provided  so  that  a 
comparison  of  any  two  outputs  of  the  three  elements  can  be  made. 
The  comparator  circuit  operates  the  switches  so  that  a  position 
is  located  where  the  outputs  again  agree  after  one  clement  fails. 

If  comparison  and  switching  are  failure-free,  the  system  will  be 
successful  as  long  as  two  or  three  elements  are  successful.  In 
this  case  the  reliability  expression  becomes: 


In  a  practical  case,  however,  failure-free  switching  cannot  be 
assumed,  and  conditional  probabilities  of  switching  operation 
must  be  considered.  Consider  the  probability  of  the  comparator 
and  switches  failing  in  such  a  manner  that  the  switches  remain 
in  their  original  positions.  If  this  probability  is  q  ,  then: 


R=Pafb+(paPc+pbPc-2Wc)(1  -V 


The  complete  problems  involve  the  consideration  of  each  of  the 
various  possible  modes  of  switching  failure  together  witn  the 
necessity  for  specific  switching  requirements  in  the  event  of 
each  element  failure.  This  is,  obviously  a  complex  problem 
and  is  beyond  the  scope  of  this  discussion.  Complete  coverage 
of  this  and  other  complex  modeling  problems  are  presented  in 
literature  referenced  at  the  end  of  this  chapter. 


6.  7  Redundancy  and  System  Reliability 


Redundancy  provides  a  means  for  improving  system  reliability 
over  that  achievable  by  a  nonredundant  system.  However,  the 
advantages  of  redundancy  can  vary  considerably  depending  on 
the  particular  redundant  configuration  used.  Evaluation  of  the 
reliability  of  a  redundant  system  configuration  to  provide  the 
greatest  reliability  improvement  involves  the  following: 

a.  Calculate  the  reliability  of  the  original  nonredundant  con¬ 
figuration. 

b.  Inspect  the  reliability  of  each  element  in  the  nonredundant 
configuration  to  determine  weak  links  where  redundancy 
might  prove  beneficial. 

c.  Consider  the  various  possible  redundant  element  configurations. 

d.  Calculate  and  compare  the  reliability  of  each  of  the  various 
redundant  element  configurations  under  consideration. 

Any  of  the  various  types  of  redundancy  discussed  previously  may 
be  employed  at  any  level  within  a  system.  The  greatest  gain  in 
reliability  is  often  realized  when  redundancy  is  applied  at  the 
lowest  possible  level.  For  example,  compare  the  reliabilities 
obtained  through  each  of  the  configurations  in  Figure  11  -  7.  Each 
configuration  represents  an  equipment  consisting  of  two  functional 
units  having  equal  reliabilities.  Configuration  A  is  nonredundant, 
configuration  B  is  redundant  at  the  equipment  level  (high  level  re¬ 
dundancy),  and  configuration  C  is  redundant  at  the  unit  level  (iow 
level  redundancy).  The  reliability  of  configuration  C  is  significantly 
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higher  than  that  of  configuration  B,  even  though  each  contain  the 
same  number  of  redundant  elements.  In  addition,  it  is  apparent 
that  redundancy  always  involves  a  penalty  in  the  form  of  increased 
weight,  space,  cos t  and  usually,  decreased  maintainability.  The 
decision  to  apply  redundancy  should  only  be  made  after  detailed 
analysis  to  evaluate  the  reliability  of  each  of  the  various  redundant 
configurations  under  consideration,  with  respect  to  the  weight  and 
space  requirements  and  cost  of  various  alternative  redundant  de¬ 
sign  possibilities. 

Decisions  can  then  be  made  to  introduce  redundancy  at  the  levels 
which  will  produce  the  greatest  overall  reliability  improvement  at 
least  cost,  and  within  the  practical  system  limitations. 

HUMAN  FACTOR  CONSIDERATIONS  IN  RELIABILITY  IMPROVEMENT 

The  importance  of  considering  human  factors  in  reliability  improvement 
becomes  evident  v/hen  system  reliability  is  defined  in  terms  of  Operational 
Reliability  rather  than  Hardware  Reliability.  AFR80-5  defines  Operational 
Reliability  as  ".  .  .  .  the  probability  that  an  operationally  ready  system  will 
react  as  required  to  accomplish  its  intended  mission.  .  .  .  ".  This  regula¬ 
tion  further  defines  a  complete  system  in  terms  of  ".  .  .  .facilities,  equip¬ 
ment,  material,  services,  and  personnel  required  for  its  operation.  .  .  .  ". 
Thus,  an  operational  system  includes  a  personnel  subsystem  to  provide 
certain  manipulative  and  decision-making  functions  that  are  essential  in 
achieving  the  required  operational  reliability.  Even  in  the  operation  of 
an  "automatic"  system,  human  performance  is  at  least  required  to  "push 
the  button"  and  initiate  hardware  operation. 

The  importance  of  the  human  element  to  achieve  operational  reliability 
varies  with  the  proportion  of  the  system  functions  that  require  human 
performance,  and  the  sensitivity  of  the  system  to  human  error.  In  many 
situations,  complex  actions  on  the  part  of  the  human  are  essential  and 
highly  critical  due  to  the  operational  nature  of  the  system.  In  such  case, 
it  is  possible  to  provide  features  to  "aid  the  operator,  but  mechanical 
means  for  replacing  or  substituting  for  the  operator  are  not  feasible.  In 
other  cases,  operational  requirements  are  such  that  the  actions  and  re¬ 
actions  necessary  are  far  beyond  the  limits  of  human  capability. 

Many  actions  can  be  performed  fry  either  man  or  machine  but  the  relative 
reliability  of  the  performance  may  influence  the  selection  of  method. 

Table  XI-3  lists  several  types  of  actions  that  are  usually  more  reliably 
performed  by  humans  and  other  types  of  actions  that  are  usually  more 
reliably  performed  by  machine.  In  a  particular  case,  however,  the 
most  reliable  performance  of  a  specific  action  may  not  necessarily  pro¬ 
vide  the  most  reliable  performance  of  the  total  system.  For  example. 
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reliability  improvement  achieved  by  providing  a  servo  mechanism  to 
reduce  certain  manipulation  errors  could  be  nullified  by  the  failure 
rate  of  the  mechanism.  In  other  cases,  the  gain  in  reliability  may 
not  be  justified  in  view  of  the  cost  of  the  more  reliable  machine. 

It  is  evident  that  system  reliability  is  dependent.,  to  a  large  extent  on 
the  effective  application  of  human  engineering  principles.  In  fact, 
many  of  the  reliability  trade-off  and  allocation  studies  performed  dur¬ 
ing  system  development  involve  significant  human  factor  parameters. 

7.  1  Operational  Reliability/Human  Error  Relationship 

The  relationship  between  human  error  and  system  reliability  can  be 
illustrated  by  defining  an  error  as  any  action  that  will  cause  an  out- 
of-tolerance  condition  to  exist  and,  therefore,  produce  a  system 
failure.  Thus,  the  probability  of  human  error  is  analogous  to  the 
probability  of  hardware  failure  in  so  far  as  operational  reliability 
is  concerned.  The  critical  nature  of  human  performance  in  relation 
to  system  operational  reliability  becomes  evident  when  the  per¬ 
sonnel  subsystem  is  considered  effectively  as  a  "series"  element 
of  the  total  system.  In  this  context,  personnel  subsystem  reliability 
can  be  defined  as  the  probability  of  error-free  operation  over  the 
required  operating  time  period,  and  could  be  included  as  a  multipli¬ 
cation  factor  in  the  total  system  reliability  equation.  Thus,  the 
overall  system  reliability  is  limited  by  the  reliability  of  the  personnel 
subsystem. 

7.2  Reliability  Improvement  Through  Reduction  Of  Human  Error 

Engineering  activities  performed  for  the  purpose  of  reducing  human 
error  and,  thereby,  improve  operational  reliability  involve  virtually 
all  aspects  of  system  design.  This  includes  the  obvious  areas  of 
operational  controls  and  other  items  in  the  man- machine  interface. 
However,  other  areas  are  also  important  considerations.  For  ex¬ 
ample,  a  maintenance  error  can  result  in  a  subsequent  operational 
error.  Therefore,  certain  aspects  of  maintainability  design  are 
also  a  part  of  the  reliability  improvement  effort.  In  fact,  all  human 
engineering  activities  during  system  development  have,  as  their 
ultimate  objective  the  reduction  of  human  error  and,  as  such,  are  a 
part  of  reliability  improvement. 

The  wide  scope  of  associated  activities  precludes  an  in-depth  dis¬ 
cussion  of  human  engineering  techniques.  However,  the  following 
list  serves  to  summarize  some  of  the  human  factor  activities  directed 
toward  the  reduction  of  human  error  and  which  are,  therefore  a  part 
of  reliability  improvement. 


a.  Identification  of  Functions  to  be  Fulfilled  by  the  System.  This 
includes  an  analysis  to  identify  all  functions  that  can  con¬ 
ceivably  be  performed  by  human  beings,  even  though  some 
may  also  be  considered  for  machine  performance. 

b.  Performing  trade  studies  where  necessary  for  optimizing  the 
allocation  of  various  functions  to  human  or  machine  operation. 

c.  For  functions  considered  for  human  performance:  performing 
analyses  to  define  all  associated  operational  requirements, 
and  to  identify  and  evaluate  alternative  devices  to  facilitate 
such  operations.  This  includes  displays,  controls,  and  devices 
or  techniques  to  aid  the  decision-making  process. 

d.  Evaluating  display  requirements  and  establishing  criteria 
such  as : 

.  The  most  appropriate  sensory  modalities  for  receiving  each 
type  of  information  in  question. 

.  Appropriate  type  of  displays  for  providing  information  when 

and  where  needed,  and  in  a  manner  that  will  insure  reception. 
This  can  include  consideration  of  factors  such  as  display 
type,  stimulus  dimension  and  codes,  and  specific  display 
features . 

.  Optimum  display  arrangement,  both  in  relation  to  the  user, 
and  in  relation  to  other  displays  and  controls. 

.  Reasonable  hounds  for  information  inputs  to  assure  com¬ 
patibility  with  human  information-receiving  capacities. 

Time-sharing  constraints  to  avoid  degradation  of  infor¬ 
mation  reception  due  to  saturation  of  response  capability. 

e.  Evaluating  decision-making  requirements  to  assure  maximum 
reduction  of  decision  errors  by  considering: 

.  Clarity  of  decisions  that  must  be  made  in  relation  to  the 
number  of  alternate  choices,  and  the  possibility  of  "pre¬ 
determining"  decisions  in  terms  of  specified  conditions. 

.  Appropriate  use  of  the  human  decision-making  ability, 

including  trade-off  between  cost  of  pre-programming  and 
adoptive  decision-making  ability  as  means  for  responding 
to  stimuli. 
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Reduction  of  decisions  to  be  made  to  the  minimum,  com¬ 
mensurate  with  system  objectives. 

f.  Evaluating  manual  control  requirements  to  assure  reliable 
operation.  This  includes  consideration  of  factors  such  as: 

.  Clarity  and  ease  of  identification  of  controls. 

.  Compatibility  of  operation  of  controls  with  respect  to 
corresponding  display  and  common  human  response 
tendancies . 

.Suitability  of  control  type  for  given  requirements. 

.  Compatibility  of  operational  requirements  of  control 
(force,  speed,  precision,  etc.  )  and  human  capability. 

.  Arrangement  of  controls  for  optimum  use. 

g.  In  addition  to  the  activities  mentioned  above,  human  engineer¬ 
ing  is  also  concerned  with  such  factors  as: 

Requirements  of  the  communications  network,  if  any, 
with  regard  to  the  burden  placed  on  the  individuals  in¬ 
volved. 

.  Logical  grouping  of  tasks  to  be  performed. 

.  Requirements  for  time-sharing  in  performing  tasks, 
especially  during  emergencies. 

.  Provisions  for  redundancies  in  the  form  of  back-up  per¬ 
sonnel  or  machines. 

.  Training  r equirements  imposed  by  the  tasks  to  be  per¬ 
formed. 

.  Compatibility  between  work  aids  and  training  aids. 

,  Training  simulator  requirements. 

.  Suitability  of  work  space. 

.  Environmental  conditions  with  regard  to  physical  well¬ 
being  of  individuals. 
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7.  3  Evaluating  Personnel  Subsystem  Reliability  Improvement 


Recently,  a  method  has  been  developed  by  which  it  is  possible 
to  evaluate  the  effect  of  man  as  a  systems  contributor.  Although 


developed  initially  to  predict  degradation  resulting  from  the  periodic 
maintenance  performed  on  rocket  engines,  its  value  as  a  predcsign 
tool  in  evaluation  of  personnel  subsystem  reliability  is  readily  evident. 


The  technique  derives  mathematical  relationships  betv/een  quantified 
observations  of  small  segments  of  human  performance,  in  terms 
of  reliability  of  task  performance,  and  a  judged  or  predicted  value 
of  a  like  segment  of  behavior  utilizing  similar  hardware  configu¬ 
rations  applied  to  a  specific  condition.  By  combining  the  observed 
and  the  judged  event  into  a  mathematical  relationship,  it  is  pos¬ 
sible  to  predict  the  reliability  with  which  the  human  will  perform 
a  task  and,  ultimately,  the  contribution  of  the  human  element  to 
system  operation. 


The  above  approach  permits  an  analysis  of  man's  function  in  a  sys¬ 
tem,  and  a  predictive  measure  applied  to  his  performance.  It  pro¬ 
vides  the  means  whereby  existing  data  may  be  extrapolated  to  con¬ 
ditions  or  procedures  not  previously  observed  or  recorded  for  the 
purpose  of  "designing-in",  or  provisioning  for.  Human  Factors 
solutions.  The  method  is  outlined  as  follows: 


Identify  the  Tasks  to  be  Performed.  The  tasks  to  be  perfor¬ 
mance  rated  are  identified  at  a  gross  level  such  that  each  task 
represents  one  complete  operation,  such  as  "perform  functional 


check,  "  or  "prepare  for  engine  leak  check.  "  Each  task  is  made 


up  of  a  series  cf  sub-tasks  which  must  be  performed  sequentially 
in  order  to  complete  the  operation. 


Identify  the  Task  Elements.  Once  the  major  tasks  have  been 
identified  each  is  broken  down  to  the  basic  elements,  or  sub¬ 
tasks,  necessary  for  task  completion.  As  an  illustration,  the 
"prepare  for  engine  leak  check"  task  can  be  defined  in  terms  of 
sub-tasks  such  as  "connect  hose,  "  "rotate  control  valve,  "  and 
"read  gauge".  Each  task  element  involves  a  small  segment  of 
human  performance  that  may  be  assessed  in  terms  of  probability 
of  error.  Further,  evaluation  of  the  sub-task  permits  appli¬ 
cation  of  the  result  to  a  number  of  seemingly  different  tasks, 
but  which  may,  in  fact,  be  composed  of  a  number  of  similar 
sub-tasks. 


1 
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c.  Obtain  Empirical  Task  Performance  Data.  Due  to  the  elementary 
nature  of  the  defined  sub-tasks,  it  is  possible  to  obtain  empirical 
data  concerning  the  reliability  ol  task  performance.  For  ex¬ 
ample,  it  is  possible  to  devise  tests  that  will  be  sensitive  to  er¬ 
rors  in  performing  single  tasks.  Much  data  of  this  nature  is 
available  in  the  literature  as  a  result  of  previous  human  factors 
studies.  Such  data  are  generally  the  result  of  controlled  tests 
performed  under  laboratory  conditions,  however,  and  appropri¬ 
ate  revisions  to  account  for  variations  in  the  use  environment 
are  usually  necessary. 

d.  Establish  Sub-Task  Rate.  In  order  to  arrive  at  element  reli¬ 
ability,  each  sub-task  to  be  considered  is  rated  in  accordance 
with  its  level  of  difficulty  or  error  potential.  "Rate"  in  this 
context,  is  the  error  potential  with  reference  to  the  require¬ 
ments  of  the  gross  task,  the  system  or  components  on  which 
the  sub-tasks  are  to  be  performed,  the  level  of  skill  of  the 
technician  on  the  job.  The  ratings  derived  for  each  task  are 
statistically  summarized,  and  a  pooled  rating  are  assigned  to 
each  element  of  work  under  evaluation. 

e.  Develop  Regression  Equation.  In  order  to  provide  the  means 
with  which  sub-task  reliability  may  be  predicted,  the  empirical 
data  and  the  judged  rating  of  that  data  are  expressed  in  the  form 
of  a  regression  line,  or  equation,  and  tested  for  goodness  of  fit 
to  describe  the  precision  with  which  human  performance  reli¬ 
ability  may  be  predicted. 

As  an  example,  a  plot  of  the  data  from  the  original  study  took 
the  form  as  shown  in  Figure  11-8. 


Sum  of  Ratings 

Figure  11-8.  Error  Rate  vs.  Error  Potential  Rating 
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The  line  that  provided  the  best  fit  was  found  to  be  expressed 
in  logarithmic  form  as: 

LogE  =  -  2.9174  +  0.006122R 

where: 

E  =  Error  Rate 

(Error  Rate  =  1 -Empirical  Reliability) 

R  =  Pooled  Ratings  of  Error  Likelihood 

The  regression  line,  or  equation,  once  developed,  is  utilized 
to  provide  sub-task  reliability  estimates  for  which  there  are 
no  empirical  data. 

f.  Establish  Task  Reliability.  The  sub-tasks  are  identified  with, 
specific  tasks  and  reliability  estimates  are  assigned  to  the 
sub-tasks  as  derived  from  the  regression  line  previously  de¬ 
veloped.  Based  on  established  procedures,  total  task  re¬ 
liability  is  obtained  as  the  product  of  the  sub-task  reliabilities 
which  may  further  be  combined  to  provide  an  estimate  of  the 
human  contribution  to  component  and  system  performance.  Task 
element  reliabilities  as  obtained  in  an  original  study  are  shown 
in  Table  XI -4.  The  method  outlined  is  applicable  to  the  evalu¬ 

ation  of  the  performance  of  one  person  acting  alone.  However, 
where  an  element  of  operator  or  technician  back-up  is  anticipated 
for  some,  or  all,  of  the  sub-tasks,  it  is  necessary  to  account 
for  the  additional  surveillance,  or  task  redundancy,  and,  thereby 
consider  the  increase  in  task  element  reliability  that  would  result. 

8.  REFERENCES 

The  following  readily  available  publications  contain  detailed  information 
on  the  reliability  improvement  techniques  mentioned  in  this  chapter, 
and  are  recommended  for  reference  where  more  complete  discussion  of 
the  subject  is  desired.  Additional  references  are  listed  in  Chapter  12. 
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TABLE  XI -4 .  Means  and  Standard  Deviations  of  Ratings  and 
Reliability  Estimates  for  the  Task  Elements  * 


Task  Element 

Rating 

Reliability 

Estimate 

Task  Element 

Rating 

Reliability 

Estimate 

frfean 

S.D. 

Mean 

S.D. 

Read  technical  instructions 

8.3 

2.2 

.9918 

Fill  sump  with  oil 

4.3 

1.6 

.9981 

Read  time  (Brush  Recorder) 

8.2 

2.1 

.9921 

Disconnect  flexible  hose 

4.2 

2.0 

.9982 

Read  electrical  or  flow  meter 

7.0 

2.8 

.9945 

Lubricate  torque  wrench  adapter 

4.2 

2.2 

.9982 

Inspect  for  loose  bolts  and  clamps 

6.4 

1.9 

.9955 

Remove  initiator  simulator 

4.1 

1.9 

.9983 

Position  multiple  position  electrical 

6.3 

2.4 

.9957 

Install  protective  cover  (friction  fit) 

4.1 

2.2 

.9983 

switch 

Read  time  (watch) 

4.1 

2.1 

.90S  3 

Mark  position  of  component 

6.2 

2.1 

.9958 

Verify  switch  position 

4.1 

1.9 

.9983 

Install  lockwire 

6.0 

2.3' 

.9961 

Inspect  for  lockwire 

4.1 

2.1 

.9983 

Inspect  for  bellows  distortion 

6.0 

2.7 

.9961 

Close  hand  valves 

4.0 

2.6 

.9983 

Install  Marman  clamp 

6.0 

1.8 

.9961 

Install  drain  tube 

4.0 

2.1 

.9983 

Install  gasket 

6.0 

2.1 

.9962 

Install  torque  wrench  adapter 

3.9 

1.7 

.9984 

Inspect  for  rust  and  corrosion 

5.9 

2.1 

.9963 

Open  hand  valves 

3.8 

2.6 

.9985 

Install  “0"  ring 

5.7 

2.2 

.9965 

1  osition  tow  position  electrical 

3.8 

1.5 

.9985 

Record  reading 

5.7 

2.3 

.9966 

switch  • 

Inspect  for  dents,  cracks  and 

5.6 

2.4 

.9967 

Spray  leak  detector 

3.7 

2.0 

.9986 

scratches 

Verify  component  removed  or 

3.5 

2.4 

.99S8 

Read  pressure  gauge 

5.4 

2.2 

,9969 

installed 

Inspect  for  frayed  shielding 

5.4 

2.3 

.9969 

Remove  nuts,  plugs  and  bolts 

3.5 

1.7 

.99SS 

Inspect  for  QC  seals 

5.3 

2.6 

.9970 

Install  pressure  cap 

3.4 

1.6 

.9988 

Tighten  nuts,  bolts  and  plugs 

5.3 

2.6 

.9970 

Remove  protective  closure  (friction 

3.2 

1.6 

.9990 

Apply  gasket  cement 

5.3 

2.3 

.9971 

fit) 

Connect  electrical  cable  (threaded) 

5.2 

2.2 

.9972 

Remove  torque  wrench  adapter 

3.0 

1.6 

.9991 

Inspect  for  air  bubbles  (leak  check) 

5.0 

2.2 

.9974 

Remove  reducing  adapter 

3.0 

1.7 

.9991 

Install  reducing  adapter 

4.9 

1.6 

.9975 

Remove  Marman  clamp 

3.0 

1.7 

.9091 

Install  initiator  simulator 

4.9 

2.5 

.9975 

Remove  pressure  cap 

2.8 

1.8 

.9992 

Connect  flexible  hose 

4.9 

2.4 

.9975 

Loosen  .nuts,  bolts  and  plugs 

2.8 

1.3 

.9992 

Position  “zero  in"  knob 

4.8 

1.6 

.9976 

Remove  union 

2.7 

1.4 

.9993 

Lubricate  bolt  or  plug 

4.7 

2.7 

.9977 

Remove  lockwire 

2.7 

1.5 

.9993 

Position  hand  valves 

4.6 

1.6 

.9979 

Remove  drain  tube 

2.6 

1.4 

.9993 

Install  nits,  plugs  and  bolts 

4.6 

1.7 

.9979 

Verify  light  illuminated  or 

2.2 

1.6 

.9996 

Install  union 

4.5 

1.8 

.9979 

extinguished 

Lubricate  “0"  ring 

4.5 

2.5 

.9979 

Install  funnel  or  hose  in  can 

2.0 

0.8 

.9997 

Rotate  gearbox  train 

4.4 

2.0 

.9980 

Remove  funnel  from  oil  can 

1.9 

1.4 

.9997 

.. 

.  .  .  . 

-  1 

*These  values  were  obtained  from  Annals  of  Reliability  and  Maintainability, 
Volume  4,  July  19&5. 
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CHAPTER  12 


RELIABILITY  REFERENCES  AND  INFORMATION 

SOURCES 


1.  INTRODUCTION 

Information  and  data  concerning  reliability  program  management  and 
reliability  engineering  are  readily  available  fiom  a  wide  variety  of 
sources  and  cover  virtually  all  aspects  of  the  reliability  technology. 
Furthermore,  the  emphasis  being  placed  on  reliability  by  industrial  and 
commercial  organizations,  as  well  as  by  the  military  and  other  govern¬ 
ment  agencies,  is  resulting  in  a  continuing  increase  in  information  and 
data . 

As  a  result  of  the  number  of  participants,  the  amount  of  information  is 
vast  and  it  would  be  impractical  to  present  a  complete  listing  of  all  re¬ 
liability  and  data  sources.  However,  this  chapter  is  included  to  identify 
a  number  of  readily  available  documents  and  references  relating  to  the 
various  subjects  covered  in  the  notebook.  Also,  some  important  sources 
of  additional  reliability  data  and  information  are  identified  at  the  end  of 
the  chapter. 

2.  MILITARY  DOCUMENTS 

Some  of  the  important  military  documents  relating  to  military  reli¬ 
ability  programs  are  listed  below.  This  list,  which  references  docu¬ 
ments  used  by  all  branches  of  the  service  is  presented  for;  information 
purposes  only,  and  is  not  intended  to  indicate  Air  Force  policy  in  regard  to 
documents  imposed  on  Air  Force  contracts. 

2.  1  Military  Standards 

M1L-STD-721  A,  Definition  of  Terms  for  Reliability  Engineering, 

1962.  . . 

i 

MIL-STD-756A,  Reliability  Prediction  Procedure  for  Aircraft. 
Missiles,  Satellites,  and  Electronic  Equipment,  1963, 

MIL-STD-810,  Environmental  Test  Methods  for  Aerospac  and 
Ground  Equipment,  1962. 

MIL-STD- 1 05D,  Sampling  Procedures  and  Tables  for  Inspection 
by  Attribute. 

MIL-STD-414,  Sampling  Procedures  and  Tables  for  Inspection 
by  Attribute. 
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MIL-STD-78 1  A,  Test  Levels  and  Accept/Reject  Criteria  for 
Reliability  of  Nonexpendable  Electronic  Equipment. 

MIL-STD-785,  Requirements  for  Reliability  Programs  (for 
Systems  and  Equipments). 

MIL-STD-690A,  Failure  Rate  Sampling  Plans  and  Procedures, 
1965.  “  ~ . .  . ‘ . ~ 

MIL-STD-790B,  Reliability  Assurance  Program  for  Electronic 
Parts  Specifications,  1966. 

2.  2  Military  Specifications 

MIL-Q-9858A,  Quality  and  Inspection  Program  Requirements. 

MIL-H-27894A(USAF),  Human  Engineering  Requirements  for 
Aerospace  Systems  and  Equipment. 

MIL-S-38130,  Safety  Engineering  of  Systems  and  Associated 
Subsystem  and  Equipment. 

MIL-R-1  961  0,  General  Specifications  for  Reliability  of  Production 
Electronic  Equipment. 

MIL -R- 2 2 73 2,  Reliability  Requirements  for  Shipboard  and  Ground 
Electronics  Equipment. 

MIL-R-22973,  General  Specification  for  Reliability  Index  Deter¬ 
mination  for  Avionic  Equipment  Models. 

MIL-R- 23094,  General  Specification  for  Reliability  Assurance  for 
Production  Acceptance  of  Avionic  Equipment. 

MIL -R-26484A,  Reliability  Requirements  for  Development  of 
Electronic  Subsystem  for  Equipment. 

MIL-R-26667,  General  Specification  for  Reliability  and  Longevity 
Requirements,  Electronic  Equipment. 

MIL -  R-  27173,  F-eliability  Requirements  for  Electronic  Ground 
Checkout  Equipment. 
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.  3  General  Use  Military  and  DOD  Publication: 


A.FSCM/AFLCM  310-1  ,  Management  of  Contract  Data  and 
F  eports. 

AFSCM  375-1,  Configuration  Management  During  Definition 
and  Acquisition. 

AFSCM  375-4,  System  Program  Management  Procedures 
AFSCM  37  5-5,  Systems  Engineering  Management  Procedures 
AFM  66-1,  Maintenance  Management 


MIL-HDBK-21  7A,  Reliability  Stress  and  Failure  Rate  Data  for 
Electronic  Equipment. 

NAVSH1PS  94501,  Bureau  of  Ships  Reliability  Design  Handbook. 

NAVSHIPS  94324,  Maintainability  Design  Criteria  Handbook  for 
Designers  of  Shipboard  Electronic  Equipment. 


HI 08,  Sampling  Procedures  for  Life  and  Reliability  Testing 
(Based  on  Exponential  Distribution).  Department  of  Defense, 
Government  Printing  Office,  Washington  D.  C.  ,  1961. 

TR-3,  Sampling  Procedure  and  Tables  for  Reliability  and  Life 
Testing  Based  on  the  Weibull  Distribution  (Mean  Life  Criterion) , 
Office  of  the  Assistant  Secretary  of  Defense  (Supply  and  Logistics) 
Washington,  D.  C.  ,  1961. 

TR-4,  Sampling  Procedures  and  Tables  for  Life  and  Reliability 
Testing  Based  on  the  Weibull  Distribution  (Hazard  Rate  Criterion) , 
Office  of  the  Assistant  Secretary  of  Defense  (Supply  and  Logistics), 
Washington,  D.  C.  ,  1962. 


3  .  COMMERCIALLY  PUBLISHED  REFERENCE  BOOKS 

1  he  following  recently  published  reference  books  and  texts  are  repre¬ 
sentative  of  the  many  books  presently  available  to  the  reliability  pro¬ 
gram  manager  and  reliability  engineer. 

ARINC  Research  Corpor at'on,  Reliability  Engineering,  Prentice- Hall, 
Inc.,  Englewood  Cliffs,  N.J.,  1964. 

R.  E.  Barlow  and  F.  Proschan,  Mathematical  Theory  of  Reliability, 
John  Wiley  lit  Sons,  Inc.,  New  York,  1969. 

I.  Bazovsky,  Reliability  Theory  and  Practices,  Prentice- Hall,  Me., 
Englewood  Cliffs,  N.J.,  1961. 

S.  R.  Calabro,  Reliability  Principles  and  Practices,  McGraw-Hill 
Book  Company,  New  York,  1962. 

D.  N.  Chorafas,  Statis tic al  Processes  and  Reliability  Engineering, 

D.  Van  Nostrand  Co.,  Inc.,  Princeton,  N.  J.  ,  I960. 

E.  L.  Grant,  Statistical  Quality  Control,  3rd  Ed.,  McGraw-Hill  Book 
Company,  New  York,  1964. 

G.  J.  Hahn,  S.  S.  Shapiro,  Statistical  Models  in  Engineering, 

John  Wiley  &  Sons,  Inc.  ,  New  York,  N.  Y.  ,  1  967. 

D.  R.  Lloyd,  M.  Lipow,  Reliability:  M anagement,  Methods  and 
Mathematics,  Prentice- Hall,  Inc.,  Englewood  Cliffs,  N.J.,  1962. 

R.  H.  Myers,  K.  L.  Wong,  H.  M.  Gordy  (cds.)  Reliability  Engineering 
for  Electronic  Systems,  John  Wiley  &  Sons,  Inc.,  New  York,  1964. 

E.  Pieruschka,  Principles  of  Reliability,  Prentice- Hall,  Inc., 
Englewood  Cliffs,  N.J.,  1963. 

G.  H.  Sandler,  System  Reliability  Engineering,  Prentice  -  Hall,  Inc., 
Englewood  Cliffs,  N.J.,  1  963. 

R.  L.  Wine,  Statistics  for  Scientists  and  Engineering,  Prentice- Hall, 
Inc.,  Englewood  Cliffs,  N.J. 
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4.  SPECIALIZED  REFERENCES 


The  articles  and  publications  referenced  be.  nv  provide  spe.  li/.ed 
information  and  data  relating  to  various  ubjects  discuss.  I  in  ibis 
notebook.  These  references  are  grouped  ac  ore  ng  to  ral  sub¬ 

ject. 

4.  1  Reliability  Program  Management 

L.  W.  Bail,  Reliability  Management  by  Objc'  lives  .no  Results, 
Proceedings,  Eighth  National  Symposium  on  Reliab  lily  ind 
Quality  Control,  1962,  pp,  156-162. 

L.  W.  Ball,  Management  Policies  for  Assigning  Departmental 
Reliability  Responsibilities,  Industrial  Quality  Control,  ASQC, 

Vol.  17,  April  1961,  pp.  16-19. 

V.  J.  Bracha,  Analysis  of  Reliability  Management  in  Defense* 

Ind ustries  ,  BSD-TDR-62-48,  June  1962. 

E.  F.  Dertinger,  Funding  Reliability  Programs ,  Proceedings, 

Ninth  National  Symposium  on  Reliability  and  Quality  Control, 

1 963,  p.  16. 

W.  R.  Kuzmin,  Rework  Costs  Related  to  Reliability  Requirements, 
Proceedings,  Sixth  National  Symposium  on  Reliability  and  Quality 
Control,  1960,  p.  95. 

1,1.  C.  Ro.mig,  PERT-PEP  Reliability  Controls  Techniques  Simplified, 
Proceedings,  Eighth  National  Symposium  on  Reliability  and  Quality 
Control,  1962. 

't 

RAW.  Smiley,  Military  Management  of  Missile  Quality  Control 
Programs ,  Proceedings,  Ninth  National  Symposium  on  Reliability 
and  Quality  Control,  1963. 

\ 

4.  2  Reliability  Assurance 

C.  J.  Brzesjinski,  Reliability  Assurance  Provisions  in  Spec  if  Ra¬ 
tions,  Industrial  Quality  Control,  Vol.  18,  No.  10,  April  1962, 
pp.  9-11. 


E,  J.  Brieding,  Purchasing  Reliability,  IRE  Transactions  on 
Reliability  and  Quality  Control,  Vol.  RQC-9,  April  I960,  pp.  19-22. 


C.  C.  Peterson,  Specification  and  Assurance  of  Large  MTBF's 
T/pical  of  Spacecraft  Electronic  Equipments,  Military  Systems 
Design,  April  1963,  pp.  27-33. 

A.  R  Park.  Reliability  Through  Adequate  Specification,  Fifth 
National  Symposium  on  Reliability  and  Quality  Control,  January 
1959,  pp.  246-250. 

R.  L.  Lander  .  Reliability  and  Product  Assurance,  Prentice- Hall, 
lnc.;  Engli  >.000  Cliffs,  N.  J.,  1963. 

AHINC  Research  Corporation,  The  Allocation  of  System  Reli- 
ability,  Publication  152-2-274,  November  1961. 

A.  P.  Basu,  Estimates  of  Reliability  for  Some  Distributions 
Useful  in  Life  Testing,  Technometrics,  Volume  6,  p.  215,  1964. 

H.  L.  Harter,  Some  Aspects  of  Reliability  and  Life  Testing, 

El<  Ironies  Division  ASQC,  Volume  3,  No.  1,  p.  5,  1964. 

G.  R.  Herd,  Some  Statistical  Concepts  and  Techniques  for  Reli¬ 
ability  Analysis  and  Prediction,  Proceedings,  Fifth  National 
Syn  posium  for  Reliability  and  Quality  Control,  1959. 

4.3  Reliability  Achievement 

B.  J.  Flehinger,  Reliability  Improvement  Through  Redundancy 
at  Various  Systems  Levels,  IRE  National  Convention  Record, 

Part  6,  1  958. 

F.  Proschan,  F.  A.  Bray,  Optimism  Redundancy  Under  Multiple 
Constraints,  Office  of  Technical  Services,  Washington,  D.  C.  , 

May  1963,  AD-403393. 

F.  E.  Dreste,  Circuit  Design  Concepts  for  High  Reliability, 
Proceedings,  Sixth  National  Symposium  on  Reliability  and 
Quality  Control,  January.  1 960,  pp.  121-133. 

L.  A.  Aroian,  R.  II.  Meyers,  Redundancy  Considerations  in 
Space  and  Satellite  Systems,  Proceedings,  Seventh  National  Sympo 
s  ium  on  Reliability  and  Quality  Control,  1961. 


L.  A.  Aroian,  The  Reliability  of  Serial  Systems  and  Redundant 
Systems,  Proceedings,  Tenth  National  Symposium  on  Reliability 
and  Quality  Control,  1964. 


I  A.  Desk,  Reliability  Considerations  in  Microminiaturization, 
NAREM  Record,  Northeast  Electronics  Research  and  Engineerin 
Meeting,  IEEE,  November  1964. 

T.  B.  Lewis,  Techniques  for  Ac  hie" '’-.g  Operational  R  liab  il  ity 
and  Maintainabilit  vr  in  Digital  Computer.  Proceedings,  Fifth 
National  Convention  Military  Electronics,  1961. 

SOURCES  Ci  REf  1  ABILITY  DATA  AND  INFORMATION 

Table  XII- 1  lists  number  of  Government  agencies  and  tc  clinical  and 
professional  societies  from  which  up-to  date  reliability  information 
and  data  can  b<  obtained. 
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Name  and  Address  Acronym  or  Otiicr  Identification 


Name  and  Address  Acronym  or  Other  Identification  Comments 


Name  and  Address  Acronym  or  Other  Identification  Comments 


INDEX 


A 


Accelerated  environments,  11-7 
Accept- reject,  6-lb 
Acceptance  tests ,  2-4 
,  Achieving  R,  2-4,  2-5,  4-1,  12-6 

Achievement  incentive,  3-13 
Acquisition  phase,  3-3,  3-18,  3-26, 
contract,  6-3 
development,  3-19,  4-16 
engineering  activities,  4-22 
initiation,  3-19 

Advanced  development  models, 
non-system  procurement,  5-21 
Advanced  Development  Objective 
(ADO),  3-5 

Advantages  of  redundancy,  11-47 
Air  Force 

contracts,  12-1 
data  policy,  5-1 
data  requirements,  5-1 
documents,  1-2. 
engineering  activities 
conceptual  phase,  4-3 
definition  phase 
phase  A,  4-7 
phase  B,  4-12 
phase  C,  4-15 

generated  reliability  data,  5-15 
policy,  12-1 

program  activities  (operational 
phase),  3-27 
requirements,  1-2 
Allocation,  1-2,  2-3,  3-10,  4-8,  8-2 
activities,  3-19 
advantages,  8-1 
analysis,  2-6 
failure  rates,  8-10 
gross,  8-4 

i^1  subsystem,  8-7 
manager  performed,  8-2 
MTBF  conversion,  8-4 
probabilistic  calculation,  8-4 


procedures,  3-10 
reliability,  3-5 
subsystem,  8-12 
system,  8-4 
study,  9-1,  9-3 
techniques,  8-2,  8-13 
Analysis 

based  on  AFM  66-1,  7-9 
degradation,  8-5 
failure  data,  3-28 
high  system  failures,  3-28 
reliability,  2-4 
Analytic  techniques,  2-1 
Apportionment,  2-3,  6-4 
failure  rates,  8-11 
MTBF,  8-8 
subsystem,  8-8 
system,  8-8 
technique,  8-9 

ARINC  airborne  equipment  method  for 
^prediction,  9-24,  9-37 
ARINC  ground  equipment  method  for 
R  prediction,  9-24,  9-37 
Assurance 

activities,  2-3 

assuring  R  program  effectiveness, 
1-2,  6-1,  6-25 

R  assurance,  2-3,  12-5,  11-9 
Atmospheric  contamination,  11-18 
Authorized  data  test,  5-2 
Average  failure  rates,  9-55 


B 

Basic 

configuration  reliability,  11-37, 
11-39,  11-40 

data  collection  of  reporting  cycle, 
7-3 
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definition  of  reliability,  9-6 
parallel  redundant  configuration, 

1  1-31 

series  system  reliability,  11-30 
Baseline 

configuration  update,  3-28 
design  requirements  update,  3-21 
documents,  3-10,  3-19 
program  requirements,  3-5,  3-11 
review  and  update,  3-8 
Bayesian  statistics 
disadvantages,  10-14 
procediires,  10-15 
techniques  and  use,  10-15 
Bird  Engineering  Research 

Associates  method  for  reliability 
prediction,  9-18 
Breathing,  11-19 
Eurn-in,  11-7 


Catastrophic  failures,  11-2 
Category  C,  R,  S,  T,  5-1  to  1 2 
Category  I  and  II  tests 

Categories  I  and  II  testing, 

3-19,  4-9 

Category  I  test  plan  and 

procedures,  3-22,  3-23 
Category  II  final  Test  Report, 
3-25,  3-28 

Category  II  test  plans,  3-18 
Responsibili'y,  3-21 
Category  I,  II,  and  III  tests  to 
produce  R  data,  6 - 1  6 
Category  III  test  and  results  and 
modifications,  9-23,  4-26,  10-2 
Certain  precautions  should  be 
observed  in  interpreting  data 
resulting  from  part  reliability 
tests,  10-37 


Collection  and  analysis  of 
field-failure  data,  2-8, 

3-29 

Commercial  "off-the-shelf" 
equipment  as  non-system 
procurement,  5-21 
Commercially  published 
reference  books,  12-4 
Commonly -us ed  techniques  for 
R  prediction,  9-16 
Comparing  reliability  of 
various  configurations, 

1 1  -48 

Comparison  of  human  and 

machine  capabilities,  11-50 
Compatibility  detail  specifica¬ 
tions  and  the  system 
specification,  3-8,  3-21 
Complex  modeling  procedures, 

8-5 

"Complexity"  of  an  item,  8-11 
Component  failure  rate,  3-1  6 
Conceptual  phase,  2-7,  3-3, 

3-4,  3-22,  8-13 
design  review,  6-11 
R  program  activities,  2-6 
Conceptual  transition,  3-3,  3-5, 
3-10,  4-5 

final  activities,  4-6 
initial  activities,  4-4 
Conditional  probabilities  of 
switching  operation,  11-47 
Conducting  SBR'n,  PDR's,  CDR's, 
and  FACI's,  6-10 
Confidence  bounds,  6-4 
Configuration  control,  3-10,  3-16, 
3-17,  4-1,  5-1 

Configuration  management,  3-2, 
3-8,  3-10,  3-19,  3-21 
Conservative  selection  and 

application  of  piece  parts,  6-8 
Considerable  overlap  between  pre¬ 
diction  and  allocation,  8-1 


Consideration  of  open-  and  short- 
circuit  failures,  11-39 
Consideration  of  subsystem 

importance  and  complexity,  8-8 
Constant  failure  rate,  9-8,  9-9 
Consumer's  risk,  10-27 
Contact  failure,  11-44 
Contract,  data  requirements  list, 

3- 12 

Contract  End  Item  (CEI)  detailed 
design,  3  -  1  6 

Contract  End  Item  (CEI)  R  ,  3-17 

4- 17 

Contractor 

budgeting  methods  for  scheduled 
design  reviews,  6-10 
II  design  organizational  structure. 
6-2,  6-14 

phase  A  of  definition  phase,  4-9 
phase  B  of  definition  phase,  4-10 
phase  C  of  definition  phase,  4-11 
R  engineering  design  reviews,  6-10 
R  program  efforts  in  proposal 
preparation,  3-13 
R  program  and  management  plan, 

3-14,  3-18,  6-7 
Cost  of  schedule,  3-7 
Counteracting  environmental  stresses, 
11-22 

Criteria  for  evaluating  the  R  factors  in 
proposals,  3-13 

Critical  Design  Reviews,  3-22,  3-24, 

4-20,  6-7 

Cri  tical  weakness  R  test,  6-18 


Data 

collection  and  analysis,  2-5 
feedback  and  analysis,  2-4 
item  and  categories  C,  R,  S,  and  T,  5-3 
obtained  during  development  and  tran¬ 
sition  stages  of  the  conceptual 
phase,  5-15 

on  plans  for  R  program,  5-19 

for  preliminary  operation  plans,  5-18 

for  pr  ocurements  not  warranting  a 

formal  system  program  approach. 

5-2,  5-23 


Data  -  -  continued 

specified  in  AFSCM/AFLCM. 

5-15 

supplemental,  7-2 
for  system  functional  description 
5-18 

Defective,  or  "weak1'  parts,  11-7 
Define 

R  design  and  trade-off  require¬ 
ments,  3-10 

R  program  milestone,  3-1  1 
R  test  requirements,  4-27 
Definition  contract,  4-6 
Definition  phase,  2-7,  3-3,  3-5, 
3-7,  3-10,  3-19,  4-6 
data  requirements,  5-15 
design  comparisons  during,  9-3 
initiation  of,  4-6 
phase  A  of,  4-6 
responsibilities  of  Deputy 

Director  for  Engineering,  4-2 
Definition  phases  band  C,  3-7 
Definition  processes,  3-2 
Definition  of  system,  equipment, 
or  component  failure,  6-1  6 
Degradation 

analysis/prediction,  8-5,  9-5, 
9-60 

oriented  testing,  11-11 
Demonstration,  2-3 

test,  from  an  accept-rejoct 
and  test-time  point  of 
view,  6-15 
total  system  R,  3-25 
Density  function,  10-4 
Deputy  Director  for  Engineering, 
3-8,  3-10,  3-12 

Deputy  Director  for  Procurement 
and  Production,  3  -8 
Deputy  Director  for  Test  and 

Deployment,  3-8,  3-11,  3-22 
Description  of  the  organization 
and  personnel  R ,  6-7 
Derated  part,  2-7 
Derating,  2-4,  11-1,  11-6 
curves ,  11-12 
disadvantage,  11-14 
figure  of  merit,  11-14 


E 


Design  Review,  2-4,  3-6,  3-12, 
6-4,  9-1 

acquisition  phase,  4-16,  9-4 
Board,  6-11 
checklist,  6-13 
component,  6-12 
conceptual,  6-12 
for  c  onceptual  design,  6-9 
manhours  required,  6-10 
plan,  6-9 

for  preliminary  design,  6-9 
for  preproduction  design,  6-9 
for  production  design,  6-9 
subsystem,  6-13 
system,  6-12 

Design  redundancy,  11-25 
Design  requirements  baseline, 
3-19,  3-21 
Designer,  6-23 
Detail  specification,  2-3 
Part  I,  3-12 
Part  II,  4-20  to  22 
Determinations  and  findings, 

(D&F),  3-5 
Development  of 

engineering  personnel  of  the 
management  regions,  6-13 
follow-on  system  modifications, 
4-26 

R  block  diagram  and  model 

during  conceptual  transition, 
4-5 

R  improvement  modifications, 
2-7 

R  requirements  for  CEI's,  3 - 1  6 
Discrimination  ratio,  10-27,  10-32 
Distribution 

of  failures,  11-3,  11-4 
gamma,  10-11 
sample  means,  10-23 
Doppler  shifts,  11-21 
Dynamic  failure,  1  1-44 


Effect  of 

accept/reject  criteria,  10-34 
consumer's  decision  risk  (g), 
10-28 

reducing  sample  size,  10-26 
sample  size  ou  R  testing,  10-21 
specified  MTBF  (6q)  and  produc¬ 
er's  decision  risk  (a),  10-29 
Effective  allocation,  8-13 
Effective  monitoring,  6-20 
Effective  R  program,  6-2 
Effectiveness  of  R  program  and 
organization,  6-24 
Electric  breakdown,  11-21 
Electromagnetic  radiation,  11-20 
Elements  impacting  on  R  program 
effectiveness,  6-2 
Elements  of  an  R  program  as  re¬ 
quired  by  data  item  R  1,  6-20 
Empirica1  R  data,  2-5 
Engineering  analyses,  2-7 
Engineering  change  proposals 
(ECP's),  4-17 

Engineering  and  R  design  reviews 
6-5,  6-7 

Environmental  stress  considera¬ 
tions  in  R  ,  3-6,  11-16,  11-17 
Equally  critical  subsystems  in 
series  configuration,  8-5 
Equipment  development  and  pro¬ 
curement,  2-1 

Equipment/system  failure  char- 
acteristics  or  distribution,  6-4 
Equivalent  AQL  values,  10-37 
Equivalent  failure  rate,  11-6 
Error  rate  versus  error  potential 
rating,  11-55 


E  stabhsning 

accept/reject  criteria,  10-34 
compatible  subsystem  R  re- 
ments,  8-10 

consumer's  decision  risk  (g) 

10-28 

minimum  acceptable  MTBF  (0  ) 
10-28  1  ’ 
quantified  R  requirements  for 
the  system,  subsystems, 
and  definable  CEI's,  3-17 
R  requirements,  8-  1 
specified  MTBF  {eQ)  and  pro¬ 
ducer's  decision  risk  (a) 

10-28 

system  and  CEJ.  R  testing  re¬ 
quirements,  J-  1 7 
Evaluating 

decision-making  requirements 
to  assure  maximum  reduction 
of  decision  errors  by  consider¬ 
ing  human  factors,  11-52 
display  requirements,  11-52 
implemented  R  plans  of  organ¬ 
izational  structure,  6-25 
manual- control,  human-factor 
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establish  sub-task,  rates,  11-55 
establish  task  R  ,  11-56 
logical  grouping  of  tasks,  11-53 
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11-12 

Incorporation  of  redundant  re¬ 
placements,  6-8 
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Information  sources,  12-1 
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Management-information  reports 
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Maximum  R,  11-16 
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Measurement,  1-1 
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Mechanisms  of  failure,  11-2 
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Military  R  programs,  12-1 
MIL-R- 27 542,  6-3 
MIL-STD-756A  AEG  Method  of 
R  prediction,  9-16 
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Multiple  parallel  redundant  con¬ 
figuration,  11-33 
Multivariate  life  tests,  11-11 


Natural  redundancy,  11-25 
Non-equivalent  subsystems  in 
series  configuration,  8-6 
Non-standard  parts,  11-9,  11-10 
Non-system  procurement  programs 
including: 

commercial  equipment  pro¬ 
curement,  4-28 
development  and  experimental 
model  procurement,  4-28 
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Pre-conditioning,  11-7,  11-8 
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Probability  of  survival,  11-29 
Procurement  activities,  3-12 
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off-the-shelf  equipments, 

5- 20 

Procurement  of  large  equip¬ 
ments  for  normal  inventory, 

3-6 

Procurement  and  production,  3-2,  5-1 
Producer's  risk  (or.),  10-27 
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management,  3-8 
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R  design  features,  3-22 
R  Design  Handbook,  2-9,  6-13 
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R  model,  3  -  i  3 
R  monitoring  and  evaluating, 

3-23 

R  Notebook,  1  -2 
R  operation  within  the  manage¬ 
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